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ABSTRACT 



This notebook contains the revised standards and 



guidelines for the National Center for Education Statistics (NCES) , 
as well as an updated and consolidated version of NCES legislation. 
It is intended for NCES staff and contractors as a guide in their 
data collection, analysis, and dissemination. These standards and 
guidelines define the limits of acceptable applications and use, to 
ensure that NCES surveys, products, and procedures become more 
uniform and have higher quality. Twenty standards and two guidances 
were adopted in January 1992 in the following areas: (1) planning and 
testing? (2) contract management and operations; (3) statistical 
processing procedures; (4) data provisions and analysis; and (5) 
evaluation and documentation. The NCES legislation is provided. 
(SLD) 
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* from the original document. 
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FOREWORD 



This document contains the revised technical standards and guidelines for the National 
Center for Education Statistics (NOES) and an updated and consolidated version of the 
NCES legislation. 

I expect that NCES standards and guidelines will be followed by all Center staff and 
contractors: 

• New Center staff should develop a working knowledge of the standards and 
guidelines, and 

• All Requests for Proposals (RFP's) must include a section that references the 
standards and guidelines and requires their use in work conducted for the 
Center. 



The Statistical Standards and Methodology Division will initiate formal evaluations from 
time to time to ensure that the standards and guidelines have been implemented in all 
phases of the Center's work and to review their operational feasibility. 

These revised standards are critical to the Centers Federal statistical responsibilities as 
an agency. They: (1) codify how we expect to behave professionally, (2) indicate the 
basis on which we expect to be judged by our peers in the statistical community, (3) 
represent the quality we expect in any of our efforts or those of our contractors and 
grantees, (4) provide a means to assure consistency among the studies the Center 
conducts, and (5) document, for users, the methods and principles the Center employs 
in the collection of data. 

I consider the revised standards and guidelines to be a visible commitment to quality in 
all of the Center's activities. 



Emerson J. Elliott 
Acting Commissioner 
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INTRODUCTION 



Purpose of Standards 

This notebook contains the revised standards and guidelines for the National 
Center for Education Statistics. It also contains an updated and consolidated version cf 
NCES legislation. It is intended for NCES staff and contractors to guide them in their 
data collection, analysis, and dissemination activities. The revised standards and 
guidelines also present a clear statement for NCES contractors and users regarding how 
data should be collected in NCES surveys, and the limits of acceptable applications and 
use. With the adoption and use of the revised standards and guidelines, expectations are 
that NCES surveys, products, and procedures shall become more uniform and of a higher 
quality. 

Background of Standards and Guidelines 

The development of formal standards at NCES began in 1S74 with the passage 
of Public Law 93-380. With this law Congress established the Advisory Council on 
Education Statistics (ACES) which was responsible for establishing standards to ensure 
that statistics and analyses disseminated by the Center are of high quality. In 1984, ACES 
requested the National Academy of Science (NAS) to undertake an evaluation of NCES 
surveys and statistical products. 

NAS found that written standards did not exist in NCES. NCES immediately took 
steps to develop and implement written standards. Twenty -one (21) standards were 
developed and were formally accepted by ACES on March 16, 1987 and were published 
in a booklet entitled " Standards and Policies" and were distributed throughout NCES. 

During 1989, a process to revise the existing standards was initiated. Since the 
initial standards were developed, NCES professionals have become considerably more 
knowledgeable and sophisticated on the subject of statistical and operational standards. 
It was decided that the revision should be done in-house using a bottoms-up approach. 
Staff committees established to review the standards agreed that the term "standard" 
should be defined as those procedures that must be done, and guidelines those things 
that are desirable and should be done but, may not bo fully met. There were many in- 
house reviews of the several drafts that were promulgated. Several presentations were 
made to the ACES and their collective advice was reflected in the later drafts of the 
standards and guidelines. The compendium of standards was considered as a whole by 
the Senior staff of NCES. Finally, the Center's strategy is to periodically review the 
standards and guidances, and as obstacles are overcome, many of the guidances may 
become standards. 

Status of the Standards 

There are currently twenty (20) standards and two (2) guidances adopted by the 
National Center for Education Statistics in January 1892. These are the same standards 

-vii- 
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and which were accepted by the Advisory Council on Education Statistics in September 
1991. 

Monitoring Activities and Plans 

The Center has awarded a contract to monitor the implementation of its standards 
and guidelines within the Center. The major purpose of this procurement is to design, 
construct, and implement a monitoring, evaluation, and reporting system for the NCES 
standards and guidelines. One major phase of this procurement is to establish current 
profiles for major NCES surveys and products in relation to the standards and guidelines. 
The contract will develop appropriate measures of compliance to the standards and 
guidelines and to design, develop and implement a PC-based monitoring and reporting 
system for standards and guidelines. 

Additional Information 

For additional information on NCES Standards and Guidelines, please contact: 

Emrnett L. Fleming, Jr. 

National Center for Education Statistics 

Statistical Standards and Methodology Division 

Capital Plaza Building, Rm. 408E 

555 New Jersey Avenue, N.W. 

Washington, D.C. 20208-5641 

Telephone: 202-219-1831 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR INITIAL PLANNING OF STATISTICAL SURVEYS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: 1-01-92 

PURPOSE: To provide broad initial guidance in designing and implementing surveys; 
and to provide input to the decision on whether or not to proceed with the preliminary 
design and implementation plans of a specific survey or survey system. 

STANDARD: The initial plan for developing a survey or survey system must include the 
rationale for the survey data, and discussion of: goals and objectives, related surveys and 
research, user needs, survey methodology, target population, analytic objectives and 
methods, basic time series items, implementation schedule, and preliminary cost and 
staffing estimates. This will result in an initial planning document that will be presented 
to the OC for review and a decision on whether to proceed with the design phase. 
Ideally, the initial plan and review by OC should be completed prior to an OMB budget 
request. If this is not possible, it needs to be done prior to preparation of an RFP. The 
plan must be of sufficient detail to serve as the introduction to a position paper or 
statement of work for the survey design work to follow. 

GUIDELINES: To meet this standard the plan should include the following: 

a) GUIDELINE: Justification of the survey in terms of issues and information 
needs, hypotheses to be tested or problems to be solved, stated in terms of 
specific goals and objectives. 

b) GUIDELINE: A review of related studies, surveys and reports of Federal 
and non-Federal sources to ensure that part or all of the data are not available 
from some existing source or could not be more appropriately obtained by adding 
questions to existing surveys sponsored by NCES or other agencies. 

c) GUIDELINE: A schedule to consult representatives of respondent groups 
and data users, and evidence that initial consultations have taken place. 

d) GUIDELINE: A preliminary survey design that discusses the proposed target 
population response rate goals, sample design, data collection methods, 

and methodological issues to be resolved. 
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STANDARD: 1-01-92 

e) GUIDELINE: A preliminary analysis plan that identifies analysis issues, 
objectives, major variables, and proposed statistical techniques. 

f) GUIDELINE: A preliminary publication and dissemination plan that identifies 
proposed major publications and their target audiences. 

g) GUIDELINE: A preliminary time schedule that accounts for the complete 
survey cycle from planning to data release. 

h) GUIDELINE: A preliminary independent government cost estimate that 
includes staffing and other costs. 



RELATED STANDARDS 

o NCES Standard for the Design of a Survey, I-02-92 
o NCES Standard for Maintaining Data Series, IV-05-92 



NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR THE DESIGN OF A SURVEY 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: I-02-92 

PURPOSE: To provide sufficient specificity for the various aspects of a survey to permit 
substantive and technical review and comment. 

STANDARD: The survey must be designed to achieve the target parameters shown 
below. If the goal is to produce separate State estimates, the design parameters apply 
to each State. A document must be developed prior to any data collection that delineates 
the basic design of a survey or survey system. The document must address the 
objectives of the survey as indicated in the initial planning document, how the objectives 
will be met by the outcomes of the survey, and the personnel resources, funds, and time 
needed to achieve the desired level of data quality, i.e. accuracy, reliability, validity. This 
document, as revised following comments, will serve as the basis for field testing and, 
once revised on the basis of field test results, will form the core of the specific tasks 
section of the statement of work for data collection. 

DESIGN PARAMETERS: 

a) Response rates for universe surveys of at least 95 percent if the data are not 
going to be weighted, and 90 percent, if the data are to be weighted. 

b) A response rate of at least 90 percent for longitudinal sample surveys (for the 
base year ana each follow-up) and at least 85 percent for cross-sectional sample 
surveys; if the activity is a survey system consisting of a cross-sectional survey 
with a longitudinal followup of a subsample, the target response rates are to be 
at least 85 percent for the base year and 90 percent for each followup; 

c) Within any stratum of a sample, the response rate must be at least 85 percent 
for longitudinal surveys and 80 percent for cross-sectional surveys; and 

d) The desired target item response rate for each critical variable must be at least 
90 percent. 
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STANDARD: 1-02-92 



GUIDELINES: To meet this standard, the documentation should include the following: 

a) GUIDELINE: A refinement of objectives of the survey or survey system such 
as: 

a. The information needs that will be met; 

b. The content area coverage; 

c. Target population (s); 

d. Primary methods of analysis; 

e. Primary user requirements; 

f. Analytic and policy purposes; and 

g. Uses in NCES compendia. 

b) GUIDELINE: An outline of the sample design plus indications of how it will 
provide sufficient coverage to meet the objectives of the survey. The outline 
should consider, if appropriate to the proposed survey, the following: 

a. Identification of the sampling frame; 

b. Sampling strata; 

c. Sample size by stratum; 

d. Expected yield by stratum; 

e. Estimated efficiency of sample design; 

f. Expected reliability of key estimates; and 

g. Appropriate variance estimation procedures. 

c) GUIDELINE: An identification of all survey data items and how these items can 
best be measured, (e.g. through questionnaires, tests), and evidence that these 
items can be measured both accurately and reliably. 

d) GUIDELINE: An analysis plan that provides strong evidence that the basic 
information needs that justify the study can be met, either descriptively or 
analytically. The plan should demonstrate how the proposed sample, the survey 
items, and measurement methods are related to the objectives of the survey. 

e) GUIDELINE: Consideration of the multiple analytic uses of the data by NCES 
and by the outside users beyond provision of basic estimates. This includes the 
use of complex analytic techniques, various levels of analysis, etc. The plan should 
consider provision of appropriate weights and calculation of appropriate variances 
for these uses. 
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STANDARD: 1-02-92 

f) GUIDELINE: If the mode of measurement is through a survey questionnaire, 
develop a set of specifications for the questionnaire or use a previous 
questionnaire. Discussion of how the items will yield the desired information and 
will minimize response burden should be presented. A plan for testing the 
questionnaire should be identified (Standard for Testing Survey Systems: 1-03-92) 
and should demonstrate conformity to the general time frame of the survey and 
allow sufficient time for redesign of the questionnaire (based on test results). 

g) GUIDELINE: A description of the anticipated data collection procedures that will 
yield data with the desired level of quality. This section should encompass: 

a. Timing of data collection; 

b. Primary mode of collection; 

c. Methods for handling non-response and/or inadequate response; 

d. Methods for testing the data collection methodology. 

h) GUIDELINE: General requirements for the data processing system, in 
terms of both hardware configurations and appropriate software. How these 
requirements are consistent with the general design of the study, meet user 
specifications, and conform to the Standard for Timely Processing and Release of 
Data Tapes should be indicated. It is understood that these requirements may vary 
by contractor and that contractor input is necessary to satisfy this guideline. 

i) GUIDELINE: A plan for preserving the confidentiality of the data that considers 
NCES* legislated mandate, the questionnaire design, analysis, data systems, files, 
and costs. 

j) GUIDELINE: A plan for quality control of each phase of the survey process 
which will permit monitoring and assessing the performance of that phase as it is 
being implemented. The plan should include procedures for modifying any design 
parameters including response rates that are not meeting expectations and testing 
modifications before changes are made in on-going survey procedures. It is 
understood that these requirements may vary by contractor and that contractor 
input is necessary to satisfy this guideline. 

k) GUIDELINE: An outline of the general parameters for evaluating survey 
procedures and results that conform to the Standard for the Evaluation of 
Surveys. 
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STANDARD: 1-02-92 



I) GUIDELINE: General specifications for an internal project management 
system that identifies (1) critical activities and key milestones of the survey 
that will be monitored: (2) the time relationship among them; and (3) a strategy for 
monitoring them and maintaining the timelines of the study. 

m) GUIDELINE: Internal estimates of cost and other resources needed by 
components of the study, including the pilot test and the evaluation. 

n) GUIDELINE: Documentation should account for the impact of the sampling error 
on the interpretation of the survey. Consideration should also be given how to 
reduce sampling error in the survey, and this should be appropriately documented. 

RELATED STANDARDS / GUIDELINES 

o NCES Standard for Initial Planning of Statistical Surveys, 1-01-92 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR TESTING SURVEY SYSTEMS 

EFFECTIVE DATE 1/1/92 NCES STANDARD: I -03-92 

PURPOSE: To ensure that all components of a survey system will function as intended 
when implemented in the full-scale survey. This includes all phases, from initial universe 
designation through weighting and files creation. 

STANDARD: Components of a survey system need to be successfully demonstrated 
through previous work or pretested prior to implementation of the full-scale survey. The 
design of a pretest must reflect conditions anticipated to occur in the full-scale study, 
particularly those likely to pose difficulties for the survey. If the guidelines for this standard 
cannot be met due to timing or budget problems, the proper course of action will be 
determined based on consultation with the Chief Statistician. Documentation of the 
pretest needs to include the design of the pretest, a description of the procedures 
followed, analysis of the extent to which the survey components met the pre-established 
criteria, discussion of other potential problems uncovered during the pretest, and 
recommendations for changes in the design to solve the problems. 

GUIDELINES: To meet this standard the following should be followed: 

1. For a new survey, the entire survey system should be pretested. Elements to be 
tested and measured, which may include alternative approaches to accomplishing 
a particular task, could include: 

a. Frame development 

b. Sample selection 

c. Questionnaire design 

d. Data collection 

e Response rates - sample and item 

f. Data processing (e.g. entry, editing, imputation) 

g. Estimation (eg. weighting, variance computation) 

h. File creation 

i. Tabulations 
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STANDARD: 1-03-92 



2 For an ongoing survey, elements being changed as a result of the evaluation of 
the survey (See Standard for Evaluation of Surveys, V-01 -92) and new elements 
or content should be pretested. 

3 Th e evaluation criteria for a successful pretest as well as methods for measuring 
the pretest results against the criteria should be developed prior to the 
implementation of a pretest. Key evaluation criteria are established during the 
design stage and should be reviewed by the Associate Commissioner of the 
division of origin. If the criteria were not met, this would mean that a survey 
component should not be implemented without pretesting a redesigned component 
in some fashion. 

4. The results of a pretest should be available and analyzed prior to making a 
decision to implement the full-scale survey and enough time should be allowed in 
planning the survey for this to occur. 

5 Survey design and instrumentation should be revised to reflect modifications 
necessary due tc the results of the pretest. A revised budget should be developed 
to reflect both changes in design and knowledge gained during the pretest about 
resource requirements. 

RELATED STANDARDS 

o NCES Standard for Initial Planning of Statistical Surveys, 1-01-92 
o NCES Standard for the Design of a Survey, I-02-92 
o NCES Standard for Evaluation of Surveys, V-01 -92 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR EDUCATIONAL TEST 

EFFECTIVE DATE 1/1/92 NCES STANDARD: 1-04-92 

PURPOSE: To ensure that educational tests produced by NCES meet professional 
standards of integrity. 

STANDARD: NCES adopts as its standard the "primary standards" specified in 
Standards for Educational and Psychological Testing (American Psychological 
Association, 1985) and the four areas specified in the Code of Fair Testing Practices in 
Education (Joint Committee on Testing Practices, American Psychological Association, 
1988). The primary standards apply to performance tests, questionnaires (such as rating 
scales), and behavioral observations. The four areas apply to developing and selecting 
tests, interpreting scores, striving for fairness, and informing test takers. 

GUIDELINES: To meet this standard, educational testing should conform to the 
following: 

1 . What the test measures and what the test is used for should be defined. 

2. The procedure used to develop tests should be clearly specified. 

3. Test questions and related materials should be reviewed and revised to avoid 
potentially insensitive content or language. 

4. Evidence of validity should be presented for each recommended use of the test. 

5. The characteristics, usefulness, and limitations of the test for the intended 
purpose should be accurately represented. 

6. Evidence of reliability and standard errors of measurement should be 
presented. 

7. Evidence should be presented concerning the appropriateness of the test for 
groups of different racial, ethnic, or linguistic backgrounds that are likely to be 
tested. Procedures should help insure that differences in performance are 
related primarily to skills under assessment rather than to irrelevant factors. 

9 



STANDARD: 1-04-92 



8. Standardized test administration, scoring, and reporting procedures should be 
followed. 

9. Special testing conditions should be arranged for special populations of 
examinees, when feasible. The comparability of resulting scores should be 
eyplored. 

10. Score reports should provide easily understood, clear, and accurate 
descriptions of test performance. The meaning and limitations of reported 
scores should be explained. 

1 1 . Users should be warned to avoid specific, reasonably anticipated misuses of 
test scores. 

12. The rights of test takers should be protected. Test takers should be informed 
how long scores will be kept on file and to whom and under what 
circumstances test scores will or will not be released. 

RELATED STANDARDS 

o NCES Standard for Initial Planning of Statistical Surveys, 1-01-92 
o NCES Standard for Testing Survey Systems, I-03-92 
o NCES Standard for Maintaining Confidentiality, IV-01-92 

RELATED CHECKLIST, FORMS AND DOCUMENTATION 

o Standards for Educational and Psychological Testing, American Psychological 
Association, 1985. 

o Code of Fair Testing Practices in Education, Joint Committee on Testing 
Practices, American Psychological Association, 1988. 
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CHECKLIST FOR EDUCATIONAL TESTS 



A. Evidence of Validity 

1 . When a test measures a construct, evidence should be presented that the 
test scores are highly related to that construct (include inter-item correlations, 
bases for weighting subscores, theoretical basis for the construct, and a 
description of the construct-related validity study). 

2. If a test is purported to be related to an outcome criterion, then evidence of the 
relationship between the test score and the outcome measures must be 
provided. Information should be available related to the rationale for choosing 
criteria, justification for selecting cut-scores, and a description of the criterion- 
related validity study. 

3. When test content is especially important (as in achievement tests), content- 
related evidence of validity should be provided. Clear definition for the content 
universe, domain specifications, item specifications, and a description of the 
content-related validity study should be provided. 

B. Evidence of Reliability and Standard Errors of Measurement 

1. Methods of calculating reliability coefficients and standard errors of 
measurement should be reported (include equations, and a description cf the 
reliability study). 

2. The psychometric procedure underlying the calculation of reliability indices 
should be specified (such as classical measurement theory, item response 
theory, etc.) 

3. Limitations in the reliability indices should be specified. For example, how does 
the index vary with changes over populations, the effects of adjustments for 
attenuation (e.g., due to restrictions of the measurement range) and the effects 
of speed (due to testing time limits). 

C. Test Development Procedures 

1. Domain definitions and specifications should be provided. Similarly, item and 
test specifications should be clearly reported. 

2. Item selection procedures should be indicated (including statistical item 
performance data, item bias information, and psychometric criteria for 
selection.) 
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CHECKLIST FOR EDUCATIONAL TEST CONT: 

D. Test Administration 

1 . Administration procedures should be standardized to reduce unwanted variation 
in test scores. The same administration manuals, training and testing conditions 
conditions should be used. 

2. Steps should be taken to insure test security, proper distribution and collection 
of materials, and a minimum of respondent burden. 

E. Scoring and Reporting 

1. Scoring procedures should be standardized, documented and checked for 
accuracy. 

2. The analysis of item responses or test scores should be clearly described ( such 
as scaling, norming, equating, etc.) 

3. Score reports should be promptly reported in a standard format that is easy to 
read. 

4. The generalizability and limitations of reported scores should be presented. 

F. Special Testing Conditions 

1. Testing modifications should be provided for examinees with handicapping 
conditions or language differences. 

2. When feasible, the validity, reliability and other indices of test integrity should 
be investigated for special populations. 

G. Rights of Test-Takers 

1. Proper consent should be obtained prior to test administration. 

2. Rights of confidentiality should be guaranteed for all uses of the data. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U. S. DEPARTMENT OF EDUCATION 



SUBJECT: GUIDANCE FOR DEVELOPING A REQUEST FOR PROPOSAL (RFP) 
FOR SURVEYS 

EFFECTIVE DATE: 1/1/92 NCES GUIDELINE: 11-01-92 



PURPOSE: To assist NCES staff to prepare an RFP containing information needed for 
an offerer to provide a proposal demonstrating technical and managerial 
competence for the successful completion of all phases of surveys. Preparing the 
RFP is only one element in the contract acquisition process. The proposal 
selection criteria are established in the RFP, but the actual process of selection 
is governed by Federal and Department of Education regulations. 

a) GUIDELINE: FRAMEWORK FOR RFP ACTIVITIES 

a.1) RFP activities are governed by guidelines established in the Federal 
Acquisition Regulations (FAR) and in the Education Department General 
Administrative Regulations (EDGAR). The Grants and Contracts Service (GCS) 
in the Office of Human Resources and Administration (OHRA), Department of 
Education is responsible for the acquisition process for NCES and can provide 
expertise and guidance in the development of the RFP. 

a.2) Within NCES, a staff member will be designated to develop a stateof work 
and related documents and will most likely also be designated the CONTRACTING 
OFFICER'S TECHNICAL REPRESENTATIVE (COTR). The individual designated 
as COTR is to be included in the development process, to provide familiarity with 
the contractual requirements and expectations. The COTR of the contract is 
responsible for (1) monitoring technical performance of the contractor, and (2) 
providing technical advice and recommendations to both the contractor and the 
Contract's Officer (CO). 
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GUIDANCE; H-01-92 

b) GUIDELINE: MAJOR COMPONENTS IN THE RFP PROCESS 

1. STATEMENT OF WORK (SOW) 

2. PROCUREMENT ACTION REQUEST (PAR) 

3. INDEPENDENT GOVERNMENT COST ESTIMATE (IGCE) 

4. PROPOSAL EVALUATION PLAN 

5. PROPOSAL PREPARATION INSTRUCTIONS PACKAGE 

6. DEPARTMENT CLEARANCES 

b.1) Prepare a STATEMENT OF WORK (SOW) 

Purpose: To provide a clear, precise, and accurate description of the 
requirement for the work and the expected activities, services, products, and 
level of effort to be delivered under the contract. 

Contents : The SOW will contain technical, managerial, and product 
specifications. The technical specifications will provide the offerer with a 
statement of statistical, temporal, resource, and format parameters of all 
phases of design, implementation, and analysis. Managerial specifications 
will be written as specific activities and tasks, delineating those to be 
performed by the contractor and those to be performed by NCES. The 
products (e.g., analysis plans, final reports) will be termed deliverables and 
each deliverable will have a time-line schedule. 

Source for contents : The Initial Planning Document, and all subsequent 
design, testing, statistical evaluation, and implementation information. 

b.2) PROCUREMENT ACTION REQUEST (PAR) 

Purpose : To obtain a funding commitment from the appropriate NCES 
budgetary sources for the survey's contractual obligation. 

Contents : Funding request and justification for funding. 

Source for contents : Office of the Commissioner, NCES. Also see samples 
from Contract Archives. 
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b.3) INDEPENDENT GOVERNMENT COST ESTIMATE (IGCE) 

Purpose : To estimate the cost of the project in terms of labor categories 
(the contractor's manpower commitment), other related costs, and to 
express the expected level of effort. 

Contents : Estimated costs for all phases and elements of the survey system 
for which the offerer will be responsible for providing the labor and 
materials. (Total estimated cost is to equal NCES budget amounts.) 

Source for contents : SF60 and previous comparable project estimates in 
Contract Archives. 

b.4) PROPOSAL EVALUATION PLAN 

Purpose : Provides the criteria on which the COTR and the Technical 
Evaluation Panel (TEP), who act as advisors to the Contracting Officer, 
assess the proposals. 

NOTE : THIS PLAN IS THE FOUNDATION FOR ALL AWARD 
DELIBERATIONS ON ALL PROPOSALS RECEIVED. IT SPECIFIES THE 
MEMBERSHIP OF THE TEP AND WHAT QUESTIONS WILL BE 
CONSIDERED IN RECOMMENDING THE AWARD OF^CONTRACT. 

Contents : Only criteria published in the RFP may be used to evaluate 
proposals submitted in response to it. Listed are those factors (e.g., 
technical competence, analysis plan, familiarity with data files, management 
plan, etc.) the Project Officer/COTR, with the approval of the CO, 
determines essential for evaluation. The number of criteria and the weights 
ascribed to each of the criteria are determined by the COTR. 

Source for Contents : The Initial Planning Document, other projects of 
comparable size and scope, and samples from the Contract Archives. 

b.5) PROPOSAL PREPARATION INSTRUCTIONS PACKAGE 

Purpose : This tells the offerer the substantive, format, and organizational 
requirements for delivering the proposal. 

Contents : The offerer will submit two separate proposals: (1) technical and 
(2) business. They are evaluated separately. 
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S ol e of Contents : For thb technical proposal, the Initial Planning 
Document and related technical and managerial concerns. For the business 
proposal, level of effort and length of contract period. 

b.6) See ACS Departmental Directive, C:GPA:2-105, Acquisition Planning 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U. S. DEPARTMENT OF EDUCATION 



SUBJECT: GUIDANCE FOR MONITORING AND DOCUMENTING SURVEY 
CONTRACTS. 

EFFECTIVE DATE: 1/1/92 NCES GUIDANCE: II-02-92 

PURPOSE: To ensure that contract progress and completion of NCES surveys are 
performed in compliance with all Federal and Department of Education regulations and 
in compliance with all terms and conditions of the contract, and that a record of the 
contractual activities is available. 

a) GUIDELINE: FRAMEWORK FOR MONITORING AND DOCUMENTING THE 

CONTRACT 

a.1) Monitoring the contract and monitoring the survey are two separate events 
(See Standard for Documenting a Survey System, V-02-92 and Standard 
for Managing Survey Operations, 1 1 -03-92). The nature of the contract will 
govern the extent of the monitoring required, but typically the Contracting 
Officer's Technical Representative (COTR) is to ensure that the contractor 
meets: (a) specifications, (b) schedules, (c) NCES standards, (d) 
performance cost controls, and (e) the beneficial effort/method of 
performance criteria in fulfilling the contract. Education Department Directive 
C:GPA:2-1 1 0 dated 1/1 2/87 established the Standards and Guidelines for 
the Monitoring of Contracts . Key elements are outlined below. 

a.2) Documenting the survey contract means maintaining up-to-date records of 
contractual activities and deliberations resulting in changes to the contract 
or to the survey. The documentation file is subject to NCES audit. 

b) GUIDELINE: MAJOR MONITORING AND DOCUMENTING ACTIVITIES FOR THE 

COTR 

1. PROGRESS REPORTS 

2. IDENTIFICATION OF POTENTIAL OR ACTUAL PROBLEMS 

3. REVIEW VOUCHERS 

4. REVIEW DELIVERABLES 

5. REVIEW CONTRACTOR PERSONNEL CHANGES 

6. ADHERENCE TO SCHEDULES 
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7. CONTRACTS ARCHIVE 

8. RECORD SCOPE OF WORK CHANGES 

9. RECORD LATE AND UNACCEPTABLE DELIVERABLES 

b.1) PROGRESS REPORTS 

According to the conditions established in the contract, the contractor will 
submit periodic (usually monthly) progress reports to the Contracting Officer 
(CO). The COTR is to review and verify the report and recommend 
acceptance to the CO. 

b.2) PROBLEM IDENTIFICATION AND RESOLUTION 

The CO and NCES management are to be advised by the COTR of any 
problem that would substantially affect the successful completion of the 
survey or alter the terms and conditions of the contract (e.g., cost or time 
increases, quality decreases). The COTR is expected to offer options and 
recommend, if possible, a solution to the problem. 

b.3) RECOMMEND ACCEPTANCE OF VOUCHERS 

The contractor will forward vouchers for payment to the CO. The COTR is 
to review the vouchers, verify the claimed costs, and recommend 
acceptance to the CO. 

b.4) REVIEW DELIVERABLES 

The COTR is to review the draft deliverables (with revision by the 
Contractor, if required) for accuracy and completeness (compliance with the 
conditions of the contract). The COTR recommends acceptance of the 
deliverables to the CO. 

b.5) CHANGES IN CONTRACTOR PERSONNEL 

If the contractor should make changes in key personnel (as defined in the 
• contract document), the COTR is to recommend to the CO approval or 
disapproval of the proposed change. 

b.6) ADHERENCE TO SCHEDULES 

For all contract and survey activities, the COTR is to monitor the timeliness 
of any scheduled event, note delays, recommend to the CO acceptance of 
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any substantive schedule changes, and notify NCES management of 
expected or actual delays. 

b.7 FORWARD CONTRACT FILE TO CONTRACTS ARCHIVE 

Upon completion and/or termination of the contract, the COTR will forward 
to the NCES Contracts Archive those items specified in the Standard for 
Documenting a Survey System, V-02-92 and Standard for Survey 
Documentation in Center Reports, V-03-92. 

b.8 RECORD ALL SCOPE OF WORK CHANGES 

The COTR should review and immediately submit all scope of work change 
requests to the Contracts' Officer for review and decision. The COTR should 
place a copy of the signed formal contract change statement in the contract 
file. 

b.9 RECORD LATE AND UNACCEPTABLE DELIVERABLES 

The COTR should keep accurate and complete records of contractor 
performance. Specifically, all late and unacceptable deliverables should be 
recorded and made part of the contract file. Furthermore, actions taken by 
the COTR or CO to remedy the problem should be documented. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR MANAGING SURVEY OPERATIONS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: H-03-92 

PURPOSE: Surveys and analyses by contract conducted by or for NCES must meet 
specifications for data collection, editing, tabulations, and quality control procedures. 

STANDARD: All phases of a survey operation must be managed in a manner which 
allows the NCES COTR to evaluate progress towards objectives. Problems must be 
identified, evaluated, and resolved as quickly as possible to keep survey operations on 
target and within scope of the work plans. 

PROCEDURES: To meet this standard requires strict adherence to project objectives, 
milestones, specifications, and contractual requirements. 

GUIDELINES: If the contractual documents or agreement fail to include survey 
management procedures then the following guidelines should provide some guidance: 

a) GUIDELINE: Survey response rates should be monitored closely. NCES 
COTRs should have the ability to access, retrieve and analyze the monitoring 
system information to ensure conformance with requirements and to independently 
determine response rates. 

b) GUIDELINE: Monthly progress reports (which should contain 
accomplishments for the prior month plus planned work for the current and 
upcoming month) should be carefully evaluated by the COTR to ensure that 
planned work activities were performed /completed as anticipated and that costs 
are within budget. 

c) GUIDELINE: In some instances, requests for additional or modified work 
procedures may be construed as "out-of-scope" requests if they have unanticipated 
situations, such as lower than expected response rates, more intense editing or 
followup requirements, key staff resignations or a simple lack of personnel, should 
be detailed in writing by the data collection organization and/or the COTR. The 
document should contain: a) a statement of the problem, reasons for the situation, 
and proposed steps to resolve the issue; b) an analysis of any statistical 
implications for survey results; and c) any contractual implications 
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such as costs, changed dates for contract deliverables and any other factors which 
may be pertinent for negotiations. This document should be placed in the survey 
project files. 

d) GUIDELINE: NCES COTRs should review/accept/reject project deliverables in 
writing within the time frames stated in the agreement or contract. 



e) GUIDELINE: Monthly progress reports (in writing or verbally) should be 
given to the COTR and the COTR's immediate supervisor or manager. Such 
reports will include an analysis of progress, potential problems and any proposed 
solutions. ( Note: During data collection, weekly status reports indicating current 
and cumulative yields are recommended) 

f) GUIDELINE: In some instances,' the data collection organization will request 
technical redirection for unanticipated problems. Such requests might be made 
verbally or in writing and outline the issue(s) and potential options. The COTR will 
use this information in discussions with other NCES personnel in determining the 
appropriate course of action. Whatever course of action is taken, it should be 
documented and placed in the project files. 

ADDITIONAL GUIDELINES 

The extent of survey management procedures will vary by the size of operations and 
other unique requirements of each project. In general, larger projects require more 
extensive management controls. The statement of work should provide for the presence 
of relevant control systems. Listed below are some options which should be considered 
when evaluating how to meet the standards and procedures described above. 

1 . The status of each survey respondent at each stage of the survey process should 
be kept current. Critical status events include, but are not limited to, dates of 
questionnaire mailout, returns, deletions (out-of-scopes), scan editing, data entry, 
machine editing, callback(s), and addition to the final data files. The status of 
survey respondents is normally best handled by a computerized status monitoring 
system. Whatever system is selected, the COTR should request direct and rapid 
access to the information. 

Note: For recurring surveys, it may be appropriate for the Center to supply the 
data collection organization with an NCES developed status monitoring system and 
train them on usage. This could avoid unnecessary software development across 
the Center's surveys. 
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Soon after initial startup of field operations, and less frequent!/ thereafter, NCES 
COTR(s) should evaluate the quality of survey operations by comparing a sample 
of the original returned questionnaires with the edited information on the data file 
to a) ensure that editing/update procedures are being correctly implemented and 
b) to learn of any problems with reporting or questionnaire design. 

On an as-needed basis, contract monitors may have a "snapshot" taken (or 
request a copy) of fully edited records from the current master file and analyze the 
information for conformance to contract requirements and deliverables. The extent 
of the statistical analysis of a "snapshot" data base will vary with survey objectives 
for descriptive analysis. Listed below are some of the possibilities that should be 
considered. 

a. For recurring surveys from the same population, an analysis might include 
a check for comparing current year data with prior year responses (or other 
existing data sources) to analyze data quality. This will help ensure the 
validity of on-going time series prior to final delivery of the master data files. 

b. In all cases, simple crosstabulations and frequencies of discrete variables 
will normally point out internal coding inconsistencies and also provide item 
response rates. 

c Simple descriptive statistics for continuous variables will provide item 
response rates, measures of dispersion, and outliers. 

The organization's data collection management and work plans should be analyzed 
periodically to determine that contracted work is being performed as expected and 
within the negotiated time frames. 

NCES COTRs might independently develop appropriate computer programs to 
analyze and accept/reject data tape deliverables. In cases where the deliverable 
is a plan or other product (sampling plan, training guide, final report, etc.), the 
deliverable should come on diskette using NCES approved microcomputer 
packages. 

The COTR(s) should assure that software developed for weighting, imputations, 
and variance estimates is accurate. This may be done through a series of 
"checkpoints" imbedded within the program (s). Another alternative is to have the 
contractor develop the software in a series of discrete steps with appropriate 
printouts produced and reviewed by the COTR along the way. 
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7. The techniques listed above are a subset of an overall quality assurance program 
for each project. Additional components of the program may include measures of 
statistical control that ought to be imposed throughout the survey management 
process. These measures might include such items as edit failure rates by item, 
response rates by sample stratum, etc. to identify and reduce errors as soon as 
possible. 

8. Constructing a well-conceived survey management plan is merely the beginning 
of the project. As soon as the survey gets underway, you will find that tasks (NOT 
deliverables) don't necessarily stick to schedule and something requires changing. 
Consequently, survey management requires the COTR and data collection 
organization to analyze the impact of changing requirements on project timelines, 
personnel requirements and costs. This is particularly true when the survey has 
a great deal of task interdependences and potential over allocations of personnel 
time. Project management packages can provide measures for such events and 
offer the ability to analyze potential options. 

9. Specify that all programs be self-documenting. This means that the programmer 
should insert "comments" within the program(s) to describe each discrete section 
of code. 

10. Complicated processes should be programmed as a series of discrete, simple 
steps. This will provide you with "checkpoints" for tracing errors. This will help you 
trace when the error(s) may have occurred. 

11. Relationships between programs and files should be flowcharted or described in 
a separate document. This includes record layouts and file structures. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR ACHIEVING ACCEPTABLE SURVEY RESPONSE 
RATES 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: il-04-92 

PURPOSE: High survey response rates in relationship to the initial plan help to ensure 
that survey results are representative of the target population. Surveys conducted by or 
for the NCES need to be conducted in a manner which encourages high rates of 
response among the selected sample. 

DEFINITION: Survey nonresponse occurs when a respondent fails entirely to respond 
to/fill out/return a data collection instrument. It is, in effect, complete item nonresponse 
from that sample unit. 

To meet this purpose the following standards must be met: 

STANDARD 1 : The data collection, independent of collection methodology (e.g. whether 
mailed or interviewer administered, in person or by telephone) must be designed and 
administered in a manner which encourages respondents to participate. The following 
are guidelines to encourage participation: 

a) GUIDELINE: Adequate explanation of need for data, and if possible, examples 
of uses of the data in the past which benefit the respondent. These materials 
should be mailed to respondent in advancs of survey if interviewer administered, 
otherwise they should be mailed with the survey instrument. 

b) GUIDELINE? Confirmation of confidentiality of responses. 

c) GUIDELINE: Interviewer training should emphasize techniques for obtaining 
respondent cooperation - manner, follow-up skills, and knowledgeability - 
appropriate timing. 

d) GUIDELINE: In keeping with the goals of the particular data collection effort, 
respondent burden should be minimized. Questions in surveys should be clear, 
brief and unequivocal. Avoid open-ended, double-barreled, leading or loaded 
questions. Skip patterns, if absolutely necessary, should be simple for interviewers 
to administer and respondents to follow. 
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e) GUIDELINE: Prior to conducting a survey, one should, as appropriate, obtain 
support and active cooperation of influential groups, i.e. student government for 
students, administrative units for schools, influential education community leaders, 
professional organizations. 

STANDARD 2: All surveys require some follow-up to achieve desirable response rates. 
Therefore, a strategy for achieving acceptable response rates needs to be planned and 
included in the RFP or contract. While the strategy will vary from survey to survey, it 
should conform to the following guidelines: 

a) GUIDELINE: Survey procedures need to be established which provide for 
timely monitoring of responses-to-date. This allows for a follow-up that can 
commence shortly after the survey begins. 

b) GUIDELINE: Obtain as much locating information about respondents as 
possible during initial sampling ie., for a student, school address and home 
address, name of advisor, phone numbers of parents. 

c) GUIDELINE: Anticipate if response rates which are low after the initial phases 
of data collection and if further data collection on the full sample is deemed too 
costly, be prepared to take a subsample of nonrespondents and utilize a more 
intensive data collection method. This will permit a description of nonrespondents' 
characteristics and allow imputation for missing characteristics if desired. 

d) GUIDELINE: Determine a set of core or key data items to obtain when a 
respondent is unwilling to fully cooperate. These variables may then be targeted 
in follow-up so that imputation is based upon some respondent characteristics. It 
also enables an evaluation of characteristics of non-respondents vs. respondents. 

RELATED STANDARDS: 

o Standard for Initial Planning of Statistical Surveys, 1-01-92 

o Standard for the Design of a Survey, I -02-92 

o Standard for Managing Survey Operations, II-03-92 

o Standard for Evaluation of Surveys, V-01 -92 

o Standard for Documenting A Survey System, V-02-92 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U. S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR IMPUTATION OF ITEM NONRESPONSE 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: 111-01-92 

PURPOSE: To ensure that there is adequate planning for NCES key item imputation in 
sample surveys and in censuses, that appropriate imputation procedures are used, and 
that these procedures are tested and found to be valid and to yield consistent estimates. 

To achieve this purpose the following standards must be met: 

STANDARD 1: For all new surveys and new data collections of ongoing surveys, a set 
of NCES key items must be identified during the planning phase, and all appropriate 
missing data imputed for these items. Key items include all items for which aggregate 
estimates are to be published by NCES. 

a) GUIDELINE: Missing data for all key items should be imputed in all surveys, 
ongoing as well as new (except for items for whioh the response rate is too low to 
permit estimation and analysis). 

STANDARD 2: Imputation procedures must be internally consistent, must be based on 
theoretical and empirical considerations, and must make use of the most relevant data 
available. 

a) GUIDELINE: There should be empirical evidence that imputation procedures 
have produced desirable results, e.g., unbiased estimates for simple means and 
totals under the assumption that nonrespondents are similar to respondents, 
maintenance of covariance structures, etc. 

STANDARD 3: Published estimates based on incomplete data for which there has been 
no imputation must note this fact, and must be accompanied by a statement indicating 
the proportion of missing data when possible. 

STANDARD 4: All Imputed values on a data file must be clearly identified as such. 




NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR COMPUTATION OF RESPONSE RATES 
EFFECTIVE DATE: 1/1/92 NCES STANDARD: IM-02 92 



PURPOSE:To ensure that the computation of response rates is consistent across 
NCES surveys. 

To meet this purpose the following standards need to be met: 

STANDARD 1: Response rates for sample surveys are to be calculated on weighted 
data (using the sample base weights); and response rates for censuses or universes 
are based on unweighted data (or, equivalently, where the weights all equal one). 
When the sampling unit is not the unit of analysis, it is appropriate to multiply the 
sampling weight of the sampling unit by the sampling weight of the unit of analysis. 
For example, we may samp'3 institutions, but be interested in estimates that are 
functions of the number of students enrolled. Then we would multiply the sample 
weight of the institution by the sampling weight of the students enrolled. This concept 
could also apply to a census. 

STANDARD 2: Overall effective response rates (Ro) are to be calculated as the ratio 
of the number of completed interviews to the number of sample respondents drawn 
minus respondents considered to be out-of-scope. As an example, in a household 
interview, this would be the number of units sampled minus the number of vacant 
units, condemned units, or units converted from residential to business use. "Unable- 
to-contact" sample units must be subsampled if it is unknown whether the units are in- 
scope or out-of-scope and if they represent a significant proportion of the initial 
sample. If the unable-to-contact sample unit is known to be in-scope but doesn't 
respond, the unit is classified as a noninterview. For unable-to contact units for which 
it is unknown whether they are in-scope or out-of-scope, a portion of them can be 
considered out-of-scope (the ratio of the # of the weighted known out-of-scopes to the 
weighted # of units sampled less the weighted # of unable-to-contacts) for the purpose 
of calculating Ro. 

weighted # of completed interviews 

RO = . 

weighted # of units sampled - weighted # out-of-scope units 
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STANDARD 3: Item response rates (Ri) are to be calculated as the ratio of the 
number of respondents for which an in-scope response was obtained to the number of 
completed interviews for which the question (or questions if a composite variable) was 
intended to be asked. 

weighted # of respondents with in-scope response 

Rj „ . 

weighted # of completed interviews for which question intended 



PRODUCT: None 
RELATED STANDARDS: 

o NCES Standard for Initial Planning of Statistical Surveys, 1-01-92 

o NCES Standard for Testing Survey Systems, I-03-92 

o NCES Standard for Managing Survey Operations, II-03-92 

o NCES Standard for Survey Documentation in Center Reports, V-03-92 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARDS FOR CODES, ABBREVIATIONS, AND ACRONYMS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: ill-03-92 

PURPOSE: To provide uniform codes, abbreviations and acronyms for use in NCES 
data collection and processing in order to facilitate the exchange of information and to 
ensure uniformity in NCES data releases. Tnis standard requires adherence to the 
following procedures: 

a) STANDARD 1: The following Federal Information Processing Guidelines (FIPS 
PUBS) 

will be used in all NCES data releases: 
FIPS PUB NUMBERS 

5- 2 States and Outlying Areas of the U.S. 

6- 3 County and County Equivalent of the States of the U.S. and 

D.C. 

8-5 Metropolitan Statistical Areas 

9 Congressional Districts of the U.S. 

66 Standard Industrial Classification Codes (SIC) 

92 Standard Occupational Codes (SOC) 

b) STANDARD 2: OMB standards for reporting race and ethnicity will be followed 
unless an alternative is justified as part of the OMB survey clearance submission. 

c) STANDARD 3: Where appropriate, the OERI Publication Guide must be utilized. 

d) STANDARD 4: Official national, State and international abbreviations are listed on 
pages 135-153 of the Style Manual. March 1984 edition, of the U.S. Government 
Printing Office. These abbreviations shall be used where appropriate in NCES 
publications. 

RELATED STANDARDS/GUIDELINES (*) 



o NCES Standard for Initial Planning of Statistical Surveys, 1-01-92 

o NCES Guidance for Developing a Request for Proposal for Surveys, 11-01-92 
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o NCES Standard for Documenting a Survey System, V-02-92 

o NCES Standard for Survey Documentation in Center Reports, V-03-92 



(*) SSMD wilt maintain a current copy of all references. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U. S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR VARIANCE ESTIMATION 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: lH-04-92 

PURPOSE: To ensure that there is adequate planning for estimation and variance 
estimation procedures to be used in sample surveys, that appropriate procedures are 
used, and that these procedures are documented. 

To achieve this purpose the following standards must be met: 

STANDARD 1: Variance estimates shall be obtained for all key variables for which 
primary estimates will be presented. 

STANDARD 2: Variance estimators will be based on theoretical and e.TDirical 
considerations related to the sample design. 

a) GUIDELINE: Estimation should, where possible, make use of other 
data from the survey, from prior surveys, or from administrative records or 
censuses. 

b) GUIDELINE: Ratio or regression estimators should be used if they 
provide some reduction in variance without substantially complicating 
subsequent analyses. 

STANDARD 3: Variance estimators need to be designed to minimize the mean squared 
error of the resulting estimates. 

a) GUIDELINE: Taylor Series, Jackknife estimators and balanced 
repeated replication should generally be considered superior to random 
groups estimators because of their greater stability in most cases. 

STANDARD 4: When replication methods are used to produce variance estimates, the 
number of replicates should be justified if that number is less than thirty because of the 
concern for the stability of the variance estimate. 
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STANDARD 5: Subject to confidentiality constraints, the data file resulting from 
full-scale data collection will contain ail the information needed for estimation and 
variance estimation, e.g., weights, PSU codes, replicates. 

ADDITIONAL GUIDELINES: 

GUIDELINE: If a large number of statistics are being estimated and presented, 
consideration should be given to modelling variances and presenting the 
summary form of the model. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U. S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR ANALYSIS AND STATISTICAL COMPARISONS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: 1H-U5-92 

PURPOSE: To ensure that comparisons, conclusions and inferences cited in descriptive 
reports are based on appropriate statistical procedures. 

To achieve the purpose the following standards need to be met: 

STANDARD 1: The analysis of an item shall be contingent on achieving an acceptable 
level of "response" in the survey. The measure of "total nonresponse" must include both 
survey nonresponse as well as item nonresponse. 

a) GUIDELINE: A reasonable measure of "total nonresponse" is : 
total nonresponse = ( 1 - survey response rate * item response rate). 

b) GUIDELINE: This measure should not exceed .30 for any analyzed variable 
unless the representativeness of the sample is evaluated by a non-response bias 
study. 

c) GUIDELINE: If the measure is unacceptable for the whole survey but is 
acceptable for a given stratum or analytic sub-universe, analysis of the variable 
at these levels may be done. 

STANDARD 2: When estimates are compared to one another in descriptive reports, the 
conclusions stated in the text are to be supported by hypothesis tests. The level of 
significance (alpha) shall be .05. 

a) GUIDELINE: Exclusive reliance on t-tests should be avoided when other 
simple procedures, such as chi square analysis or analysis of variance, are 
appropriate. 

b) GUIDELINE: When a chi square analysis or an analysis of variance does 
not show a statistically significant overall difference among categories, it is 
inappropriate to t-test individual differences for pairs of categories. 
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STANDARD 3: Hypothesis tests shall incorporate the design effects 
associated with the estimates being tested. 

a) GUIDELINE: A permissible (although not necessarily preferred) way 
to incorporate design effects into chi square analysis and analysis of 
variance is to reduce effective sample sizes. 

STANDARD 4: When t-tests are used to compare means or proportions, multiple 
testing needs to be combined into sets, based on the nature of the conclusions to be 
drawn, and a Bonferroni (or similar) correction for multiple testing need to be 
incorporated. Multiple comparison procedures include Bonferroni, Scheffe, and Tukey 
tests. 

STANDARD 5: Multiple comparison procedures are to be used to control the level of 
type I error for simultaneous inferences. 

a) GUIDELINE: In analyses involving multiple variables, factors, or levels, an 
overall F or chi-square test of significance needs to be performed to determine 
which variables or factors are significant before conducting multiple comparisons. 
The overall test of significance could be from an analysis of variance, a two-way 
contingency table, or the test of a loglinear model in the case of higher 
dimensional tables of categorical data. When multiple comparisons are used, the 
number of comparisons and the overall type I error rate should be given for each 
simultaneous inference. 

b) GUIDELINE: The number of "simple" comparisons in a bulletin or report 
should be minimized in order to reduce the probability of a spurious significant 
difference, or type I error. A "simple" comparison is defined as a t-test, chi- 
square test, or any other test that examines a simple hypothesis such as the 
difference between two means or proportions. 

c) GUIDELINE: Consideration should be given to use of multivariate techniques 
such as regression and log-linear mpdels in analyses involving multivariable data. 
Multivariable data refer to several variables measured on each sample unit. 

STANDARD 6: When across-the-board t-testing is conducted, two-sided t-tests are to 
be used. 
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ADDITIONAL GUIDELINE: 

GUIDELINE: The above standards apply primarily to across-the board 
testing of differences. When a small number of differences are selected 
for testing, based on a priori considerations, it may not be necessary 
to use corrections for multiple testing, to use chi square analysis 
or analysis of variance prior to t-testing, or to use two-sided t-tests. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR MAINTAINING CONFIDENTIALITY 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: IV-01-92 

PURPOSE: To protect the confidentiality of NCES data that are individually identifiable. 

STANDARD: Staff will be informed about the requirements of the law and will monitor 
the confidentiality of individually identifiable information in their daily activities and in the 
release of information to the public. 

LEGAL REQUIREMENTS: 

Protection of the confidentiality of individually identifiable information collected by NCES 
is covered by two laws. One is the Privacy Act of 1974 and the other is the 1988 
Hawkins-Stafford amendments of the GEPA (PL 100-297). However, a recent technical 
amendment to GEPA has exempted NPSAS and certain other longitudinal studies from 
the Hawkins-Stafford Amendments. These exempted survey studies are still subject to 
the requirements of the Privacy Act of 1974. Individual information means any 
information about an individual. Therefore, a principal providing information about 
himself or subjective information about the school is an example of individual 
information, while a principal providing objective information about his school is not. 

Privacy Act of 1974-When individual records are indexed in any fashion subject to the 
provisions of the Privacy Act, they constitute a system of records. NCES has 
developed two types of systems of records: one for its surveys that maintain individually 
identifiable data and one for affidavits of nondisclosure. 

Program managers and system managers have legal responsibilities relating to the 
Privacy Act of 1974 to maintain these systems of records. These requirements are 
defined in an ED Privacy Directive dated 12/28/88 . 

PL 100-297 (Stafford-Hawkins) -This law requires, except for (i) the survey required 
under section 1303(c) of the Higher Education Amendments of 1986; or (ii) certain 
longitudinal studies concerning access, choice, persistence progress, or attainment in 
postsecondary education, that no person may: 

(i) use any individually identifiable information furnished under the provisions of this 
section for any purpose other than statistical purposes for which it is supplied, 

(ii) make any publication whereby the data furnished by any particular person under this 
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section can be identified; or 

(iii) permit anyone other than the individuals authorized by the Commissioner to 
examine the individual reports. Further, individually identifiable information is immune 
from legal process, and shall not, without the consent of the individual concerned, be 
admitted as evidence or be used for any purpose in any action, suit, or other judicial or 
administrative proceeding. Employees, including temporary employees, or other persons 
who have sworn to observe the limitations imposed by this law, who knowingly publish 
or communicate any individually identifiable information will be subject to fines of up to 
$250,000, or up to 5 years in prison, or both. 

PROCEDURES: In order to maintain confidentiality for NCES data collections, the 
following conditions must be met. 

1. As required by the Office of Management and Budget, respondents must be told 
whether or not participation in the survey is voluntary and about the legislative or 
executive mandate for the survey. 

2. Respondents must be told in a cover letter or in instructions that all responses 
that relate to or describe identifiable characteristics of individuals (or, in the 
special case of NAEP, schools since the legislation for NAEP treats NAEP 
sample schools as individuals) will be kept confidential. Furthermore, the routine 
statistical purposes for which the data may be shared must be explained. 
Respondents whose data cannot be kept confidential must be given similar 
information such as: the purpose of the survey, whether it's voluntary or not, the 
legislative or executive mandate, routine uses, and the extent to which, if any, 
responses will be kept confidential. 

3 Ail NCES staff, without exception, have pledged not to release, for any purpose, 
to any person not sworn to the preservation of confidentiality, any individually 
identifiable data. Individually identifiable data are confidential and protected from 
legal process unless the individual provides written consent. 

4 All contractors whose activities might involve contact with individually identifiable 
data shall provide NCES Project Officers with a list of all staff who might have 
contact with such data, together with a certificate to the effect that all such staff 
have taken an appropriate oath of confidentiality (a notarized copy of each 
affidavit of nondisclosure must be attached). These oaths will be kept current as 
new staff are assigned to NCES projects with individually identifiable information. 

5. At the discretion of the Commissioner of NCES, staff may release individually 
identifiable information to persons for statistical uses compatible with the 
purposes for which the data were collected if those persons sign oaths of 
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confidentiality and meet such other requirements as deemed necessary in 
accordance with other confidentiality provisions of the law. 

6. In reporting on surveys and preparing public use data tapes, the goal is to have 
an acceptably low probability of identifying individual respondents. 

a. For reports, this means that each publication cell must have at least three 
(unweighted) observations in it and subsequent tabulations (e.g. 
crosstabulations) must not provide additional information which would 
disclose individual identities. 

b. For public use data tapes, one must consider any variables proposed for 
inclusion on the tape that are unusual (such as very high salaries) and 
data sources that may be available in the pubiic or private sector for 
matching purposes. Details on this topic are provided in Policies and 
Procedures for Public Release of Data. 

7. In preparing tapes for release to the public, the tapes must undergo a disclosure 
analysis. (In designing and conducting this analysis, staff must consult the 
Disclosure Review Board (DRB) for guidance at an early stage.) Any 
modifications that are necessary as a result of the analysis must be made, and 
the entire process must be documented. The proposed final tape and 
documentation of the disclosure analysis will be reviewed by the DRB. The DRB 
will make a recommendation to the Commissioner of NCES about its release. In 
making its recommendation, the DRB will take into consideration information 
such as resources needed in order to disclose individually identifiable 
information, age of the data, accessibility of external files, detail and specificity of 
the data, and reliability and completeness of any external files. 

8. Both NCES staff and contractors must employ methods in data collection and 
analysis that protect the confidentiality of individually identifiable information. 
Among other things, this means that data tapes, questionnaires, and other 
reports having individual data must be kept secure at all times through the use of 
passwords, physical separation of individual identity from the rest of the data, 
and secure data handling and storage. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR TABULAR AND GRAPHICAL PRESENTATIONS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: IV-02-92 

PURPOSE: To ensure that tables and graphs used in NCES publications display data 
in such a way that readers can conveniently evaluate the accuracy of the results 
presented. To ensure consistent and accurate results from calculations. The problem of 
implying a spurious degree of accuracy must be balanced with the need to have 
consistency between and within tables. 

To achieve this purpose the following standards must be met: 

STANDARD 1: All tables and graphs must have a short title concisely stating the 
subject of the table, dated, and must be able to stand alone. 

STANDARD 2: All summary U.S. tables and graphs must present data representing 
the 50 States and the District of Columbia, if compatible. Data for the outlying areas 
are to be excluded unless separate totals are shown. 

" "ANDARD 3: For all variables used in a table, the categories must be labeled 
along with the names of the variables. 

Additional Guidance: 

o All tables must have row and column totals. 

o For tables of percentages or rates, the population must be presented in 
the table when possible. 

STANDARD 4: All tables and graphs must have source notes. When the data 
presented in a table or graph come from multiple sources or come from a source 
which is not the direct subject of the report, the source note must clearly match the data 
items with the source. 

STANDARD 5: Standard errors (se's) or confidence intervals (CI's) on statistics in 
tables can be handled in two ways. For publications that are targeted to general 
audiences, a separate table of standard errors on key statistics may be presented in the 
technical appendix. The second method for more technical publications is to include 
se's or ci's in the table being presented, either in a separate subsection of the table or 
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in columns accompanying the statistics being presented. 

STANDARD 6: Before rounding figures for publication, a decision must be made about 
the appropriate number of significant digits added after the decimal point. 

STANDARD 7: When adding, multiplying, or dividing figures that have been rounded to 
different significant digits, the product can only be stated in terms of the number with 
the fewest significant digits. 

STANDARD 8: Tabulations to produce summary data and computations performed for 
purposes of estimating standard errors must be done on unrounded numbers when 
available. 

STANDARD 9: Sums of column (row) figures in a table must be derived using 
unrounded figures, with appropriate rounding of the total after its derivation. To handle 
the problem of column (row) figures not summing to a rounded total, an explanatory 
footnote must be used. For example, "Because of rounding, details may not add to 
totals." 

STANDARD 10: The final rounded value must be obtained from the original figure 
available, not from a series of roundings (e.g. 7.1748 can be 7.175 or 7.17 or 7.2 or 7 
but not 7.18). This situation typically arises when researchers round percentages from 
tables in tenths of a percent to full percents to be used in text. 

STANDARD 11: Percentages must generally be rounded to no more than one decimal 
place. 

STANDARD 12: Specific rules to be used for rounding: If the digit to be dropped is less 
than 5, the last retained digit is not changed. 

6.1273 6.127 

If the digit to be dropped is greater than 5, the last digit retained is increased by 1 . 

6.688 6.69 

If the digit to be dropped is equal to 5, make no changes in the preceding digit if it is 
even and increase it by 1 if it is odd. 

6.845 6.84 
6.875 6.88 
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STANDARD 13: Documents need footnotes explaining cases where actual numbers are 
used in all calculations, and rounded results are presented which seem inconsistent. 

RELATED STANDARDS: 

o NCES Standard for Dissemination of Survey Data and Reports, IV-03-92 
o NCES Standard for Variance Estimation, lil-04-92 

RELATED CHECKLISTS, FORMS, AND DOCUMENTS: 

o Guidelines for Use of Graphics (Below) 
o NCES Guidelines for Tabular Presentation 

o James Filliben and David DesJardins, the Graphic Analysis and Display of 

Statistical Data, Washington Statistical Society, 1986. 
o Calvin F. Schmid and Stanton E. Schmid, Handbook of Graphic 

Presentation, 1979 

o Edward R. Tufte, The Visual Display of Quantitative Information. Graphics 

Press, 1983. 
o OERI Publication Guide published in 1990. 



Additional Guidance for Use of Graphics 

a) Simplicity is critical; cluttered graphs are ineffective. Generally no 
more than 7 segments in a pie graph. 

b) When using time-series data, horizontal axis intervals should be equal to time 
intervals. Time scales should be linear. 

c) Ordinarily, the vertical scale should start at zero. Otherwise, scale breaks 
should be clearly visible. 

d) Bar charts should not be used for showing changes over time unless only two 
or at most three points are being compared. 

e) Stacked bar graphs, stacked area line graphs, and histograms are hard to 
read and easily misinterpreted and should be avoided in most situations. 

f) All axes should be clearly labeled. 

g) The use of multiple pie charts to compare similar items is generally 
inappropriate because of the difficulty in comparing the relative sizes of wedges. 
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h) Labels should be placed on the tick marks for axes. 

i) When using black and white maps, area data should contain only one variable 
and no more than 5, preferably 3, levels of distinction. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR DISSEMINATION OF SURVEY DATA AND REPORTS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: IV-03-92 

PURPOSE: To ensure comprehensive, appropriate, and predictable dissemination of 
survey data and reports to the public. 

STANDARD: Survey data and reports must be disseminated according to a written 
plan which addresses both dissemination of machine-readable data for secondary 
analysis and dissemination of NCES and contractor reports based on survey data. 

a) . The dissemination plan must be developed in consultation with the office of 
the Commissioner for each survey during the initial survey planning process. 
The size and detail of this plan will vary with the size and complexity of the 
survey, but the plan must address both dissemination of machine-readable data 
for secondary analysis and dissemination of NCES and contractor reports based 
on survey data. 

b) .The dissemination plan must specify a process for dissemination of reports 
based on survey data. This section of the plan must address the types of NCES 
reports (e.g., E.D.Tabs, Survey, Analysis, and Technical Reports) that are 
proposed for preparation by NCES staff and contractors. The plan must describe 
the types of audiences targeted by each set of reports. 

GUIDELINES: To meet this standard, the following guidelines should be followed: 

a) GUIDELINE: Produce a one to two page descriptive brochure about the survey 
system in conjunction with the Outreach staff in IS. This brochure should describe the 
purpose of the survey, components of the survey, sample sizes, sample design, special 
features of the survey, and ordering information for data tapes and survey reports when 
available. 

b) GUIDELINE: Plans for dissemination of reports should not be limited to NCES 
publications. Staff members are strongly encouraged to prepare presentations at 
professional meetings and papers for professional journals. The dissemination plan 
should include provisions for these types of activities. 
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c) GUIDELINE: Dissemination of recurrent NCES publications (such as regular data 
series) should receive special consideration in the dissemination plan. 

d) GUIDELINE: Dissemination of Machine-Readable Data for Secondary Analysis 

The dissemination plan should address the following three aspects of 
public access: data access, conceptual access, and technical access. 

1 . Data access 

Raw data can be made available to the public in different formats, 
such as magnetic tape, CD-ROM, special extracts (on floppy disks 
or magnetic tape), and data base management systems. Each of 
these formats will enhance data access for different user groups. 
The dissemination plan should specify the data formats that are 
intended for release of survey data. 

2. Conceptual access 

In order to encourage more widespread use of NCES data, 
conceptual access should be considered in the dissemination plan. 
Conceptual access addresses the needs of potential users of NCES 
data to understand its suitability for secondary analysis. Some 
potential users (a) may not be informed of the existence of NCES 
databases, (b) may not know enough about NCES data and its 
availability to understand its potential relevance to research 
questions of interest, or (c) may not have sufficient information 
about specific NCES database contents and survey design to 
determine its suitability for studying particular research questions or 
hypotheses. Therefore, the dissemination plan should include a 
section which proposes methods to disseminate widely among such 
potential users information about the content and structure of NCES 
databases, the sample design, the various formats in which the 
data are (or will be) available, and sources of the data. 

Examples of Methods of Conceptual Access: 
Descriptive brochures about the database 
Notices and/or articles in popular education 
journals such as Educational Researcher or 
Education Week 
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Presentations at professional meetings 
Users meetings and seminars (general and 
topical) 

3. Technical Access 

Despite attaining conceptual access to NCES databases, some 
potential users of data for secondary analysis may not possess the 
technical skills and/or resources to use the data to its fullest 
potential. The dissemination plan should address appropriate 
methods to improve technical access, such as training sessions or 
individual consultation. Database documentation should contain the 
name and phone number of a contact person to aid users with 
technical questions and problems. 



RELATED STANDARDS: 

o Standard for Initial Planning of Statistical Surveys, 1-01-92 
o Standard for Maintaining Data Series, IV-05-92 

o Standard for Timely Processing and Release of Data and Data Tapes, IV-04- 
92 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR TIMELY PROCESSING AND RELEASE OF DATA AND 
DATA TAPES 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: IV-04-92 



PURPOSE: To ensure that NCES data are released in a timely and predictable 
fashion. 

STANDARD: Public-use data files, E.DTABS and survey reports are to be released by 
the scheduled target dates specified in the publication planning document, The NCES 
publication process must be followed and documented on the publication sign off sheet. 
Prior to the release of a micro data file, a report presenting the key data contained on 
the file must be adjudicated and made available to the public. 

GUIDELINE: 

In order to set realistic time lines and to identify problem areas in the production 
process, a management information system should be established to collect and 
analyze data on the elapsed times between major steps in the publication process. 
Dates for the following milestones noted below should be included in the management 
information system. 

PUBLIC USE DATA TAPES 

1 . Data collection completed 

2. Clean data tape completed 

3. Tape announcement and documentation to branch chief 

4. Branch chief initial approval 

5. Division director initial approval 

6. Review by SSMD 

6. OC approval 

7. Tape available for sale in OERI/ Office of Deputy Assistant Secretary 
for Operations/ Publication Division. 

E.D. TABS 

1. Data collection completed 
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2. Clean data tape completed 

3. E.D.TABS to branch chief 

4. Branch chief initial approval 

5. Division director initial approval 

6. Adjudication by SSMD 

7. OG approval 

8. Copies of E.D.TABS produced 

9. E.D.TABS in distribution 

SURVEY REPORTS 

1 . Data collection completed 

2. Clean data tape completed 

3. Branch chief initial approval 

4. Division director initial approval 

5. Adjudication meeting set (not date of adjudication meeting) 

6. Adjudication meeting held (date of meeting) 

7. Response to adjudication initially approved by branch chief 

8. Response to adjudication initially approved by division director 

9. Final OC approval 

10. Editor assigned 

1 1 . Editor and author agree on editorial changes 

12. Author prepares camera-ready copy and returns it to Publication Division 

13. Copies of report are produced/printed 

14. Report in distribution 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR MAINTAINING DATA SERIES 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: lV-05-92 

PURPOSE: To maintain basic data series collected by the Center for continuing and 
periodic surveys by preserving the consistency of data over time. The goal is to have 
time series data for basic items that are published on a schedule so that the public will 
know when to expect these data series from year to year. 

STANDARD: The Center must maintain and report on a consistent set of basic data 
series that may be analyzed over time. A plan detailing a list of basic data items that 
will be maintained as part of the survey along with the justification for their designation 
needs to bo incorporated into the Initial Planning Document for review and comment by 
the OC. Decisions regarding those data series and definitions must be coordinated with 
the Data Development Division. 

GUIDELINES: To meet this standard, the following guidelines should be used: 

a) GUIDELINE: Planning for the maintenance of basic data series for recurring surveys 
should be done concurrently with the initial planning of the survey. In the Initial 
Planning Document, identify the basic data items needed to address policy issues and 
other information needs to be assessed on a regular basis. 

b) GUIDELINE: Provisions should be made to maintain the same data items and 
survey procedures from survey to survey. These should include the use of consistent 
definitions for the data items and the same survey procedures, such as the method of 
data collection, data processing and tabulations, and the imputation of missing data. 

c) GUIDELINE: If changes are needed in data items or survey procedures for data 
series, provide justification or rationale for the changes in terms of their usefulness for 
making policy, conducting analyses, and addressing other information needs. Also 
adjustment methods for these changes, such as crosswalks and bridge studies, should 
be provided so that trend analysis is possible. 

d) GUIDELINE: To the extent practicable, estimates of change as well as level should 
be included in reports. For survey reports, consider publishing three or more years of 
the basic data along with the current year to highlight the time series. 
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PAGE 2 



e) GUIDELINE: In the appendix of the survey report, consider providing a list of other 
publications containing the data for previous years. 



RELATED STANDARDS: 

o NCES Standard for Initial Planning of Statistical Surveys, 1-01-92 

o NCES Standard for Dissemination of Survey Data and Reports, IV-03-92 

o NCES Standard for Machine Readable Products, IV-06-92 

o NCES Standard foi Survey Documentation in Center Reports, V-03-92 

o NCES Standard for Maintaining Confidentiality, IV-01-92 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR MACHINE READABLE PRODUCTS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: IV-06-92 

PURPOSE: To ensure that data files created by NCES staff and contractors are fully 
useable by researchers within and outside the Center. 

STANDARD: Machine readable products shall be released in formats that enable wide 
accessibility by data users. Documentation shall provide information to identify all 
variables with survey form items and explain construction of composite items. 

To meet this standard the following requirements must be followed: 

a) Data Set Names must identify the survey system, component, year of survey, 
and version number and whether the data is a raw file, SAS file or other. 

b) Tapes stored at the Department's central computer facility must be 9-track 
EBCDIC, 6250 BPI with IBM standard labels or tape cartridges. The Office of 
Educational Research and Improvement will produce tapes in different 
specifications to meet user requests. Products for personal computers must be 
MS-DOS compatible. 

c) File structure, record format, and data element conventions must be consistent 
with NCES checklist on "Data Tape Preparation." 

d) Data Base Documentation, must be completed for all tapes and disks. This 
includes: 

1 . Abstract or summary. 

2. Survey methodology including general description of data 
collection methods, weighting procedures, description of editing, 
error resolution, and imputation flags, guidelines for processing the 
data, dates of coverage, frequency counts, and response rates. 

3. Information on how to use replicate weights or PSUs and stratum 
for variance estimation. 

4. Procedures for using weights to produce estimates. 

5. File description. 

55 



STANDARD: IV-06-92 



6. Record length, format and count. 

7. Data element definitions. 

e) A machine readable file must be placed on tapes prepared for public 
distribution, along with a file description and record layout information. 

ADDITIONAL GUIDANCE 

SAS and SPSS system files with appropriate documentation must be prepared 
for each survey release with more than 1,000 respondents. 

RELATED STANDARDS: 

o NCES Standard on Confidentiality, IV-01 -92 

o NCES Standard for Timely Processing and Release of Data and Data Tapes, 
IV-04-92 

o NCES Standard for Maintaining Data Series, IV-05-92 

o NCES Standard for Survey Documentation in Center Reports, V-03-92 



RELATED CHECKLISTS AND FORMS: 

o Data Tape Preparation Checklist 

o OERI Form 27: Data Base Documentation 



Checklist 

Data Preparation 

A. Data set names should contain the survey system, component, year and ec 
(preliminary, revision 1, revision 2, final, etc.). 

B. Standard Definitions 

1. FIPS Standards are used where applicable. 

2. NCES standard definitions and codes are used where applicable. 

C. Data Element Conventions 

1 Numeric fields contain only numbers, zeros or missing values. 

2, Zeros are represented by "0". Blanks or may not be used to 
represent O's. 

3. Data should distinguish between legitimate skips and non-response. 
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4. Suppression symbols are removed from numeric fields and stored in 
associated "flag" fields. 

5. Negative numbers on data tapes are represented in standard (IBM) signed 
numeric format. 

6. When practical, numeric data fields containing continuous variables should 
be identical in length. 

7. Separate record locations are used for all data items. 

8. Imputed data should be flagged. Imputation method should be identified in 
the flag, i.e. 0= original data; 1=logical impute; 2= stochastic impute; 
3=impute from previous year; etc. If feasible, imputation cells should be 
placed on the end of the record 

D. Record Conventions 

1 . Rectangular structures are used, where possible. 

2. Single record type is used. 

3. Logical record lengths are constant. 

4. Each record contains a unique identifier such as ID. 

E. Record Layout 

1 . Record layout, location and format are documented. 

2. Blocking factors and block size are documented, where appropriate. 

3. Record counts are included. 

4. Each record type is identified. 

R Format 

1. Tapes 

a. 9 track, EBCDIC, 3480 & 6250 BPI tapes or tape cartridges are used. 

b. Standard alphanumeric characters are used. 

c. Binary or packed decimal fields are avoided unless required by the 
size of the file or other factors. 

2. Diskettes and CD's 

a. MS-DOS compatible file structures are used. 

b. Data may be in ASCII or files constructed for use in specific software 
packages. However, files should be directly accessible or able to be 
translated easily with widely available software, i.e. LOTUS, DBASE, or 
WordPerfect. 
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G. Quality Control Procedures 

1 . Record counts and totals are consistent between preliminary files 
and the final released data files or explanations are included in the 
documentation. 

2. Internal inconsistencies are corrected or documented. 

3. Major inconsistencies between current and past data are resolved or 
documented. 

4. Imputed data are flagged. 

5. Flags are consistently positioned within a file. 

6. Codes for missing, refused, and non-applicable data are consistent 
within a survey or survey system. 

7. All necessary steps have been taken to protect the confidential of 
respondents (see Standard for Maintaining Confidentiality, IV-01-92). 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR EVALUATION OF SURVEYS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: V-01-92 



PURPOSE: The results of the statistical evaluation must enable the users of the 
survey data to understand the quality and limitations of the data and must provide 
information for planning future surveys or replications of the same survey. Also the 
inclusion of a systematic assessment of all sources of nonsampling error for key 
statistics to be studied or reported in NCES publications. 

STANDARD 1: All proposed and ongoing surveys conducted by NCES must include in 
their design an evaluation plan. This plan, to be worked cut jointly with SSMD: (1) will 
identify potential sources of nonsampling error, (2) provides for the measurement of the 
magnitude of the various types of nonsampling error, and (3) includes studies designed 
to determine what factors are associated with differential levels of nonsampling error 
and to assess procedures for reducing the magnitude of these errors, and (4) funding 
for the evaluation is to be included in the budget request and appropriate activities 
included in the work statement of the RFP. This will result in the following: 

1. Statistical evaluation plan with cost estimates for each project; 
for ongoing surveys, the plan must be multiyear in nature. 

2. Technical report or series of technical reports summarizing results of 
evaluation studies. 

3. Brief statement of evaluation results, for example, nonresponse rates, 
comparisons to other estimates, to be included as part of survey reports. 

4. Recommendations for issues to be evaluated in the next replication of the 
survey or in similar surveys in the future. 

GUIDELINES: To meet this standard, the following process should be undertaken with 
the assistance of SSMD: 
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a) GUIDELINE: Review of past surveys similar to the one being planned in order 
to determine what statistical evaluation data have been collected in prior surveys 
and any potential problems which have been identified. Based on this review, a 
summary should be written on what is known about the sources and magnitude 
of nonsampling error and a list prepared identifying those statistical issues which 
should be addressed in the current survey's evaluation. The list should be 
reviewed and prioritized. 

b) GUIDELINE: For each of the issues identified as priorities in the above step, 
the plan should indicate how the issue will be addressed, including what data will 
be used from within the survey and from outside sources and how these data will 
be obtained, what kinds of comparisons will be made, what kinds of experiments 
will be built into the survey, and how the data will be analyzed. 

c) GUIDELINE: In addition to issues identified prior to the beginning of the 
survey, staff should be cognizant of other problem areas arising during the 
course of the survey and, where possible, collect and analyze appropriate data to 
assess the magnitude of the problem. 

d) GUIDELINE: Data from the statistical evaluation should, where feasible and 
appropriate, be analyzed prior to or concurrent with the analysis of the survey 
data so that the results of the evaluation can be taken into account in analyzing 
and interpreting the actual study data. 

e) GUIDELINE: The final product of the evaluation program should be a 
technical report or series of technical reports on the statistical studies carried out. 
In addition, a brief summary of the evaluation findings should be prepared for 
inclusion in the appendix of any report of the survey's findings. 

f) GUIDELINE: An additional product of the evaluation activity will be a written 
set of recommendations as to what should be evaluated in the next replication of 
the survey (next follow-up or next study in a series). 

STANDARD 2: Documentation must be made of the kinds of nonsampling errors found 
in the survey and of the procedures (if any) used to correct for them. This 
documentation must be maintained both for future design and also for reviewers and 
users to understand how the survey data were developed. 
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GUIDELINES: The following guidelines are helpful in meeting this standard: 

a) GUIDELINE: During the sample design phase for any survey conducted 
by or for the Center, each potential source of nonsampling error should be 
listed and, where possible and appropriate, methods for bounding or 
estimating the error from each source should be developed and 
implemented. 

b) GUIDELINE: Where possible, nonsampling error estimates and bounds 
should make use of data from other surveys, or from administrative 
records or censuses: any different source of information is useful as an 
indicator. 

c) GUIDELINE: For recurring surveys, a sustained effort should be made 
toward the development of an error profile document itemizing all sources 
of error. Where possible, estimates or bounds on the magnitudes of these 
errors should be provided; the total error model for the survey should be 
discussed; and the survey should be assessed in terms of this model. 

RELATED REFERENCES 

Table 1 , Attached : Examples of Typical Studies Designed to Evaluate Different 
Types of Nonsampling Error. 

Brooks, Camilla A. and Bailar, Barbara A. An Error Profile: Employment as 
Measured by the Current Population Survey. Washington: U. S. Department of 
Commerce, 1978. 

Jabine, Thomas B.; King.Karen E.; and Petroni, Rita J. Survey of Income and 
Program Participation Quality Profile. Washington, D.C.: U.S. Department of 
Commerce, Bureau of Census, 1990. 

Groves, Robert M. et al. Telephone Methods for Surveys . New York: 
Wiley, 1988. 

Groves, Robert M. Survey Errors and Survey Costs . New York: Wiley, 
1989. 

Kish, Leslie Statistical Design for Research . New York: Wiley, 1987. 
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RELATED STANDARDS AND GUIDANCE: 

o NCES Standard for Initial Planning of Statistical Surveys, 1-01-92 

o NCES Standard for Testing Survey Systems, I -03-92 

o NCES Guidance for Monitoring and Documenting Survey Contracts, II-02 

o NCES Standard for Evaluation of Surveys, V-01-92 

o NCES Standard for Documenting a Survey System, V-C 2-92. 



TABLE 1 

A CHECKLIST DESIGNED TO EVALUATE DIFFERENT TYPES OF 
NONSAMPLING ERROR 

(1) Nonresponse 
Unit Nonresponse: 

-Description of characteristics of units not responding 
-Examination of changes in response rates over time 
-Intensive followup of a sample of nonrespondents to ascertain 
nonresponse bias 

-Assessment of method of handling nonresponse: imputation or weighting 
procedures 

Item Nonresponse: 

-Description of items not completed, patterns of partial non-response, 
characteristics of sampling units failing to respond to certain 
groups of characteristics 

(2) Coverage - adequacy of frame 

-Matching studies to earlier versions of the same data source or to 
other data sources 

-Comparison of a very reliable external source for some subset of the 
current population to the current frame 
-Analysis of survey returns for deaths, duplicates, changes in 
classification, and out of scope units 
-Field work - such as area listings 
-Review of frames by states, associations, LEAs 
-Comparison of estimated counts with estimated counts from another 
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source. 

(3) Validity - accuracy of responses 

-Re-interview Study with adjudication of disagreements 
-Site visits to examine administrative records and compare to subject 
responses 

-Comparison with outside data sources 

(4) Reliability - consistency of responses 
-Re-interview Study 

-Examination of changes in response over repeated questioning 
-Agreement among multiple respondents in a sampled unit (e.g. 
parent/student) 

- Agreement of statistics derived from different sections of the questionnaire 
or different questionnaires. 

(5) Survey Design Issues 

-Pretests to determine the efficacy of devices to improve response 
(e.g. offering incentives) 

-Pretests to compare alternative ways of collecting or processing data 
-Comparison of final design effects with estimated design effects used in survey 
planning 

-In a longitudinal survey, establishment of a small independent 
replicate of the sample for use as a test group while the survey 
continues to allow an evaluation of the effect of changes 

(6) Estimation 

-Examination of the choice of estimator/design 

-Possibility of fitting survey distributions to known distributions from other 
sources to reduce variance and bias 

-Re-estimation using alternative techniques, such as alternative outlier 
treatments, alternative imputation procedures, and alternative 
variance estimation techniques 

-Analysis of the imputation process - frequency of imputation, initial 
and final distributions of the variables 
-Use of generalized variance curves 

-Effect of changes in data processing procedures on survey estimates 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR DOCUMENTING A SURVEY SYSTEM 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: V-02-92 

PURPOSE: To ensure that complete documentation is kept on the Center's projects 
and their associated deliverables. 

STANDARD: The documentation of a survey system must include the history of a 
project (via the maintenance of records) from its initial planning stages to its final 
evaluation, in sufficient detail so that major events, decisions, and changes can be 
reconstructed and studied. In addition to recording the decisions that were made, it is 
equally important to document why decisions were made. Similarly, the conceptual 
basis for the methodology is as important as the methodological specifications. It will 
result in survey system documentation maintained in an accessible and secure place. 
This documentation will serve as the basis for assessing standards compliance, final 
methodological reports, data file documentation, and future survey planning. 



GUIDELINES: To meet this standard, the following procedures should be followed: 

a) GUIDELINE: For each project, one or more individuals) should be designated 
to be responsible for keeping records on the project. 

b) GUIDELINE: Documentation should include detailed records on the planning, 
design, development, implementation, and evaluation phases of the project. The 
records should include changes in the project and the reasons for these changes 
as well as a) project planning documents, including issues analyses; b) sampling 
plans; c) correspondence; d) editing specifications; e) progress reports and 
vouchers; f) OMB clearance package materials; g) any contract 
addenda/changes that affected the scope of work or technical redirection; and h) 
contract deliverables. 

c) GUIDELINE: For each project, a separate log book and/or data file containing 
the names of all individuals who have been involved with the project should be 
kept. For each person on the list, documentation should contain name, address, 
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affiliat' in, phone #, and role that person played or is currently playing. This list 
will also serve as a mailing list. This log book should also record the major 
day-to- day communication of the project. Verbal decisions concerning the 
project should be documented and included in log. 

d) GUIDELINE: Semi-annually, each Center project should be reviewed for 
completeness by the branch chief and division director. A written review 
(complete with suggestions) should be added to file and also presented to 
individual responsible for record keeping. 

e) GUIDELINE: Documentation of any part of the project should be aimed at 
several different groups including management, technical staff, planners of other 
surveys, and outside users requiring documentation of varying levels of 
complexity. 
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NATIONAL CENTER FOR EDUCATION STATISTICS 
OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT 
U.S. DEPARTMENT OF EDUCATION 



SUBJECT: STANDARD FOR SURVEY DOCUMENTATION IN CENTER REPORTS 

EFFECTIVE DATE: 1/1/92 NCES STANDARD: V-03-92 

PURPOSE: To provide in each Center report the appropriate amount of documentation 
on the survey data, methodology, and other important aspects of a survey. 

STANDARD: All Center reports must include documentation which allows the reader to 
understand the nature and limitations of the results presented. The level of detail 
included will vary depending on the type of report. The general areas to be covered 
include: (1) an abstract; (2) current status of data (preliminary, revised or final) and 
relation to previous surveys in series; (3) goals of the-survey; (4) methodology; and (5) 
guidelines for using the data. Specific items of information to be included in each of 
these components in each type of report are provided in the Guidelines section below. 

GUIDELINES: To meet this standard the following should be considered: 

a) GUIDELINE: Survey documentation should enable the reader - even the 
non-statistical user - to understand its contents, and the use and limitations of 
data readily and clearly. 

b) GUIDELINE: The attached table outlines the types of documentation to be 
included in the various types of Center reports. An "X" indicates the item is to 
included; a "B" indicates that a brief description should be included "-" indicates 
the item is not necessary. 

RELATED STANDARDS: 

o NCES Standard for Initial Planning of Statistical Surveys, 1-01-92 
o NCES Standard for Machine Readable Products, IV-06-92 
o NCES Standard for Documenting a Survey System, V-02-92 
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GLOSSARY FOR NCES STANDARDS 



(Adopted from the Standards for Education Data Collection and Reporting) 



Abstracting - the process of converting information from existing recording into a 
format required for analysis. 

Alternative Hypothesis - see hypothesis. 

Analytic Techniques - techniques which address questions about how or why units in 
the population are related (see descriptive techniques). 

Archiving - a process of rotating copies of files used in various phases of data 
processing through a secure area. 

Audiences - persons and organizations who will be guided in making decisions by the 
results of the data collection activity and all others who are interested in the results of 
the data collection. 

Base Weight - the product of all component weights; accounts for all stages of 
sampling units. 

Bias (due to nonresponse) - difference that occurs when respondents differ as a 
group from nonrespondents on a characteristic being studied. 

Bias (of an estimate) - the difference between the expected value of a sample 
estimate and the corresponding true value for the population. 

Burden - the aggregate hours realistically required for data providers to participate in 
a data collection. 

CAPI - Computer Assisted Personal Interviewing enables data collection staff to use 
portable microcomputers to administer a data collection form while viewing the form on 
the computer screen. As responses are entered directly into the computer, they are 
used to guide the interview and are automatically checked for specified range, format, 
and consistency edits. 

CATI - Computer Assisted Telephone Interviewing uses a computer system that 
allows a telephone interviewer to administer a data collection form over the phone 
while viewing the form on a computer screen. As the interviewer enters responses 
directly into the computer, the responses are used to guide the interview and are 
automatically checked for specified range, format, and consistency edits. 
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GLOSSARY FOR NCES STANDARDS CONTINUED 

CD ROM - Compact Disc Read Only Memory is a computer storage disk in the same 
physical form as an audio CD. A CD Rom disk can store approximately 550 
megabytes of digital data. 

Census - a count of ail the elements of a population and a determination of the 
distributions of their characteristics. 

Closed-Ended - a type of question in which the data provider's responses are limited 
to given alternatives. 

Codebook - a record of each variable being measured, including variable name, 
columns occupied by each variable in the data matrix, and values used to define each 
variable. 

Coding - the act of categorizing raw data into groups or giving the data numerical 
values. 

Component Weight - for each stage of sampling, the component weight is equal to 
the reciprocal of the probability of selecting the unit at that stage. 

Computer Assisted Personal 'nterviewing - see CAPI. 

Computer Assisted Telephone Interviewing - see CATI. 

Confidence Interval - a sample-based estimate expressed as an interval or range of 
values within which the true population value is expected to be located (with a 
specified degree of confidence). 

Consistency Edits - see logic edits. 

Construct - a concept that describes a characteristic, attribute, or variable 
relationship. The concepts are often unobservable ideas or abstractions such as 
community context or performance. 

Correlation - the tendency for certain values or levels of one variable to occur with 
particular values or levels of another variable. 

Correlation Coefficient - a measure of association between two variables that can 
range from -1.00 (perfect negative relationship) to 0 (no relationship) to + 1.00 (perfect 
positive relationship). 

Data Dictionary - a database that holds the name, type, range of values, sources, 
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and authorization for access for each data element in an organization's files and 
database. It may also indicate which application programs use that data so that when 
a change in data structure is contemplated, a list of the affected programs can be 
generated. 

Data Element - the most basic unit of information. In data processing it is the 
fundamental data structure. It is defined by its size (in characters) and data type (e.g. 
alphanumeric, numeric only, true/false, date) and may include a specific set of values 
or range of values. 

Data File Backup - copies of the latest data files that can be used to restore lost 
data. 

Data Flow Diagram - depicts the movement of data within a system by describing the 
data and the manual and machine processing performed on the data. 

Descriptive Techniques - techniques that address questions about how many units in 
the population surveyed have a certain characteristic. 

Design Effect - the variance of an estimate divided by the variance of the estimate 
that would have occurred if a sample of tne same size had been selected using simple 
random sampling. 

Development Stage - the third of four stages in the development of a data processing 
system; entails writing computer programs that follow the systems design 
specifications. 

Field Test - the study of a data collection activity in the setting where it is to be 
conducted. Emphasis is on the place, not the method. 

File - a set of related records. 

File Corruption - a file ihat has been physically damaged or that contains errors on 
the magnetic surface that prevent the file from being accessed by a computer. 

File Design - the method used by a file to store and retrieve data 

File Recovery - copying the file from a current backup version. 

File Regeneration - the process of running necessary software to rebuild the file to its 
current form. 
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Final Adjusted Weight - the product of all base-weight components, nonresponse 
adjustments, and the poststratification factor, if any. 

"Hot Deck" Imputation - a process that replaces missing data items with values 
observed from other sampled cases. 

Hypothesis - an assumption about a property or characteristic of a population. In 
statistical theory, there are usually two hypotheses under consideration, and the goal 
is lo decide which of the two hypotheses is likely to be true. The null hypothesis 
usually corresponds to the hypothesis to be tested by accepted statistical techniques. 
That is, the null hypothesis is considered to be true unless there is compelling 
evidence from the sample data that it is false. The alternative hypothesis is the 
complement of the null hypothesis. 

Imputation For Item or Survey Nonresponse - substituting plausible values for 
missing values in the survey data set. 

Inferential Techniques - see analytic techniques. 

Input - any data entered into a computer or the act of entering data by keyboard, light 
pen, mouse, graphics, table, magnetic disk or tape, communications channel or 
key-punch card. 

Item Nonresponse - items on a data collection from that are missing when a 
response was expected. 

Linearity - a relationship in which, when any two variables are plotted, a straight line 
results. 

Logic Edits - checks made of the data to ensure common sense consistency among 
the responses from a data provider. 

Master File - a collection of records holding the current data in an information system. 
It contains descriptive data as well as summary information. 

Modeling - a general class of statistical techniques in which the mathematical 
relationships between several variables are developed and analyzed. 

Multiple Comparison Procedures - statistical procedures that take into account the 
fact that several statistical tests (a family of tests) are being performed simultaneously. 
In particular, multiple comparison procedures give the proper type I errors on an 
experimentwise or familywise basis. 
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GLOSSARY FOR NCES STANDARDS CONTINUED 

Nonresponse - cases in data collection activities in which potential data providers are 
contacted but refuse to reply or are unable to do so for reasons such as deafness or 
illness. 

Nonresponse Bias - occurs when respondents as a group differ from nonrespondents 
in their answers to questions on a data collection form. 

Nonsampling Error - an error in sample estimates that cannot be attributed to 
sampling fluctuations. Such errors may arise from many sources including imperfect 
selection, bias in response or estimation, and errors of observation and recording. 

Null Hypothesis - see hypothesis. 

Open-Ended - a type of interview question that does not limit the potential responses 
to predetermined alternatives. 

Operational Definition - the sequence of steps or procedures a researcher follows to 
obtain a measurement; specifies how to measure a variable. 

Operationalize - to describe constructs or variables in concrete terms so that 
measurements can be made. 

Optical Disk - a disk that is read optically (e.g., by laser technology) , rather than 
magnetically. 

Out-of-Range Response - a response that is outside of the predetermined range of 
values considered acceptable for a particular item. 

Outliers - an observation so far separated in value from other observations that it 
raises the question of whether it comes from a different population or whether the 
sampling technique is at fault. 

Output - any computer-generated information appearing on hard copy, video display, 
or machine readable form (e.g., disk or tape). 

Pilot Test - a brief and simplified preliminary study designed to test methods and to 
learn whether a proposed data collection activity appears likely to yield valuable 
results. 

Population - all individuals in the group to which conclusions from a data collection 
activity are to be applied. 
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Population Variance - a measure of dispersion defined as the average of the 
squared deviations between the observed values of the elements of a population or 
sample and the corresponding mean of those values. 

Poststratification Adjustment - a weight adjustment that forces survey estimates to 
match independent population totals within selected poststrata (adjustment cells). 

Precision - the difference between a sample-based estimate and its expected value. 
Precision is measured by the sampling error (or standard error) of an estimate. 

Pretest - a test to determine performance prior to the administration of a data 
collection activity. 

Probability Sample - a sample selected by a method such that each unit has a fixed 
and determined probability of selection. 

Processing - the manipulation of data. 

Range Check - a determination of whether responses fall within a predetermined set 
of acceptable values. 

Record Format - the layout of the information contained in a data record (includes the 
name, type, and size of each field in the record). 

Records - a logical grouping of data elements within a file upon which a computer 
program acts. 

Regression - a statistical technique in which the functional relationship between a 
dependent variable and one or more independent variables can be estimated. 

Reliability - the consistency in results of a test or measurement including the 
tendency of the test or measurement to produce the same results when applied twice 
to some entity or attribute believed not to have changed in the interval between 
measurements. 

Replication Techniques - methods of estimating sampling errors that involve 
repeated estimation of the same statistic using various subsets of data providers. The 
two primary methods are balanced repeated replication (BRR) and the jackknife 
technique. 

Sample - subgroup selected from the entire population. 
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Sampling Error - that part of the difference between a value for an entire population 
and an estimate of that value derived from a probability sample that results from 
observing only a sample of values. 

Sampling Strata - mutually exclusive and exhaustive subsets of the population within 
which elements of the population have similar characteristics, to the extent feasible. 

Sampling Variance - a measure of dispersion of values of a statistic that would occur 
if the survey were repeated a large number of times using the same sample design, 
instrument, and data collection methodology. The square root of the sampling 
variance is the standard error. 

Simple Hypothesis - a statistical hypothesis that completely specifies the distribution 
function of the variates concerned in the hypothesis. 

Special Population - a subset of the total population distinguishable by unique needs, 
characteristics, or interests (e.g., disadvantaged students, gifted students, physically or 
mentally handicapped students, vocational education students). 

Standard Deviation - the most widely used measure of dispersion of a frequency 
distribution. It is equal to the positive square root of the population variance. 

Standard Error - the positive square root of the sampling variance. It is a measure of 
the dispersion of the sampling distribution of a statistic. Standard errors are used to 
establish confidence intervals for the statistics being analyzed. 

Statistically Significant - there is a low probability that the result is attributable to 
chance alone. 

Symmetry - refers to a property of a relationship in which no distinction is made 
between independent and dependent variables; 'asymmetry" refers to a property of a 
relationship in which such a distinction is made. For example, the Pearson 
product-moment correlation coefficient is a symmetric measure of linear association 
between two intervally scaled variables while the linear regression coefficient is an 
asymmetric measure for such variables. 

System Flowchart - shows, in addition to the flow of data, the relationship of 
macro-components of the system. 

System Operator - a person who is responsible for the physical operation of the 
equipment (e.g., CPUs, disk drives, tape drives, printers). 
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Systems Design Stage - the second of four stages in the development of an 
information processing system, it expands upon the systems requirements specified in 
the Systems Requirements Stage. 

Systems Requirements Stage - the first of four stages in the development of an 
information processing system, it defines the scope of the project. 

T-Test - any statistical test in which the underlying distribution of the test statistic has 
a Student's t-distribution. Most commonly, t-tests are used to compare the means of 
two subsets of a population under study. 

Testing Stage - the fourth of four stages in the development of an information 
processing system, it provides for the testing of all computer programs according to a 
test plan before the system is considered operational 

Transaction File - a collection of records that record any change in a master file. 
The data in transaction files are used to update the data in master files. Transaction 
files also serve as audit trails and, after a period of time, are transferred from on-line 
disk to off-line tape for future statistical and historical processing. 

Type I Error - in a formal statistical test of a hypothesis, the probability of rejecting the 
null hypothesis when it is true. 

Type I! Error - in a formal statistical test of a hypothesis, the probability of accepting 
the null hypothesis when it is false. 

Unit Nonresponse - failure of a survey respondent to provide any information. 

Validation Sample - a sample drawn from a study so that the survey responses can 
be compared with values from another source that is assumed to contain the correct 
values. 

Validity - the capacity of a measuring instrument to predict what it was designed to 
predict; stated most often in terms of the correlation between scores on the instrument 
and measures of performance on some criterion. 

Variance - see population variance and sampling variance. 

Verification - checking the work performed by another person for accuracy. 



76 



GLOSSARY FOR NCES STANDARDS CONTINUED 

Weighted Estimates - estimates from a sample survey in which the sample data are 
weighted (multiplied) by factors reflecting the sample design. The weights (referred to 
as sampling weights) are typicality equal to the reciprocals of the overall selection 
probabilities, multiplied by a nonresponse or poststratification adjustment. 
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this Act (relating to the National Aatmment of Education 

P3 ^&ot > lea& than $8,750,000 shall be available in each Steal 
year to carry out section 406 of this Act, except for subsection 
(i) of that section (relating to the Center for Educational Statis- 
tics); and 

(F) not less than $500,000 shall be available in each fiscal 
year to carry out subsection (dX4XAXiii) of this section (relating 
to field initiated research). 
(2) If the sums appropriated for any fiscal year are less than the 
amount required to be made available under subparagraphs (A) 
through (F) of paragraph (1), then each of the amounts required to 
be made available under such subparagraphs shall be ratably re- 
duced. If additional amounts become available for any such fiscal 
year, such reduced amounts shall be increased o*< the same basis as 
they were reduced. 

(20 US.C. 1221c) Enut«3 June 23, 1972, PX. 92-S18, kc 801(eX2i. 86 Stat. 528, 
232; amended August 21, 1974, PX. 93-S&Q. ssc. &020>X2XB), S8 Stat W0; amended 
October 12, 1976, PX 94-482, t*c 403, 90 Stat 2227, 2228, 2229, 2230; amended Nov. 
1, 1978, PX 95-561, sec. 1242, 92 Stat 2352, 2353; see alto finer*' reference Oct. 17, 
1979, PX 96-S8, tec 301, 93 Stat 677; amended Oct 8, 1980 9 PX 9&-S74, sec. 1811- 
1314, 94 Sut 1498, 1499; amended Oct 19, 19S4, PX 98-511, sees. 702(a). 703, 704(a). 
98 Sut 2405. 2406; amended Oct 17, 1986, PX 99-498, |1401(a). 100 Sut. 1589; 
amended June 8, 1987, PX 100-50, {24(a). 101 Sut. 862; amended April 28, 1988, 
PX 100-297, sees. 8001(|>), 8002, 8403(c), 102 Sut 837, 849. 



Sec. 406. (aXD There is established, within the Office of Educa- 
tions] Research and Improvement, a National Center for Education 
Statistics (hereafter 1 in this section referred to as the "Center"). 
The general design and duties of the National Center for Education 
Statistics shall be to acquire and diffuse among the people of the 
United States useful statistical information on subjects connected 
with education (in the most general and comprehensive sense of 
the word) particularly the retention of students, the assessment of 
their progress, the financing of institutions of education, financial 
aid to students, the supply of and demand for teachers and other 
school personnel, libraries, comparisons of the education of the 
United States and foreign nations and the means of promoting ma- 
terial, social, and intellectual prosperity through education. 

(2XA) The Center shall be headed by a Commissioner of Educa- 
tion Statistics who shall be appointed by the President, by and 
with the advice and consent of the Senate. The Commissioner of 
the National Center for Education Statistics shall have substantial 
experience and knowledge of programs encompassed by the Nation* 
al Center 2 The Commissioner shall be paid in accordance with sec* 
tion 5315 of title 5, United States Code. The Commissioner shall 
serve for terms of 4 years, except that the initial appointment shall 
commence June 21, 1991. 

fB) There shall be within the Center (i) an Associate Commission- 
er for Statistical Standards and Methodology who shall be qualified 
in the field of mathematical statistics or statistical methodology; 
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and (ii) an Associate Commissioner for Data Collection and Dis- 
semination, who thrf] be an individual who has extensive knowl- 
edge of uses of statistics for policy purpose* at all level* of Ameri- 
can education, and who shall promote the participation of btates, 
localities, and institutions of higher education in designing educa- 
tion statistics programs, encourage widespread dissemination ana 
use of the Center's data, and promote United States Mrticipation 
in international and regional education statistics. The Coxnmiasion- 
er may appoint such other Associate Commiasioners as may be nec- 
essary and appropriate". 

(b) The purpose of the Center shall be to collect, and analyse 1 
and disseminate statistics and other data related to education in 
the United States and in other nations. The Center shall— 

(1) if feasible, on a State-by-State basis, collect, collate, and, 
from time to time, report full and complete statistics on the 
conditions of education in the United Statea; 

(2) conduct end publish reports on specialized analyses 01 the 
meaning and significance of such statistics; t a _ 

(3) assist State and local educational agencies,, including 
State agencies responsible for postsecondary education, in im- 
proving and automating their statistical and data collection ac- 
tivities (and shall establish a special program to train employ- 
ees of such State and local agencies in the use of the Center * 
standard statistical procedures and concepts and may establish 
a fellows program to temporarily appoint such employees as 
fellows at the Center for the purpose of f a miliar ization with 
the operations of the Center); . , . _ 

(4) review and report on educational activities in foreign 
countries; m9 ... , , 

(5) conduct a continuing survey of institutions of higher edu- 
cation and iocal educational agencies to determine the demand 
for, and the availability of, qualified teachers and administra- 
tive personnel especially in critical areas within education 
which are developing or are likely to develop, aad assess the 
extent to which programs administered in the Department ot 
Education are helping to meet the needs identified as a result 
of such continuing survey, and 

(6) access 2 periodically the current and projected supply and 
demand for elementary and secondary school teachers (includ- 
ing teachers at the preschool level) and early childhood educa- 
tion development personnel with particular attention to— 

(A) long-term and short-term needs for personnel in vari- 
ous subject areas or teaching specialities: 

(B) shortages in particular types of schools or communi- 
ties, and instates or regions; 

(C) the number of minorities entering teaching; m 

(D) the proportions of women and minorities in educa- 
tional administration, and the trends over time; 

(E) the demographic characteristics, academic qualifica- 
tions, job preparation, experience and skills of existing 
teachers and new entrants ii, the field of education; 
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(F) the effect of the introduction of State mandated 
teacher competency teats on the demographic and educa- 
tional characteristics of teachers and the supply of teach- 
ers; and 

(G) the rate at which teachers leave teaching, their rea- 
sons for leaving, the source* of supply for new entrants, 
and the trends over time. 

(cXl) There ahall be an Advisory Council on Education Statistics 
which shall be composed of ? public members appointed by the Sec- 
retary and such ex officio members as are listed in subparagraph 
(2). Not more than 4 of the appointed members of the Council may 
be members of the same political party. 

(2) The ex officio members of the Council shall be — 

(A) the Assistant Secretary, 

(E) the Director of the Census, 

{O the Commissioner of Labor Statistics, 

(D) Commissioner 1 of Education Statistics, and;* 

(E) Chairman, 9 National Commission on Libraries and infor- 
mation Science. 

(3) Appointed members of the Council shall serve for terms of 8 
years, as determined by the Secretary except that in the case of 
initially appointed members of the Council, they shall serve for 
shorter terms to the extent necessary that the terms of office of not 
more than 3 members expire in the same calendar year. 

(4) The Commissioner of Education Statistics shall serve as the 
non-voting presiding officer of the Council. 

(5KA) The Council ahall meet at the call of the presiding officer, 
except that it shall meet — 

(i) at least four times during each calendar year; and 

(ii) in addition, whenever three voting members request in 
writing that the presiding officer call a meeting. 

(B) Six members of the Council shall constitute a quorum of the 
CounciL 

(6) The provisions of section 448(b) of part D of this title ahall not 
apply to the Council established under tnis subsection. 

07) The Council shall review general policies for the operation of 
the Center and shall be responsible for advising on standards to 
insure 4 that statistics and analyses disseminated by the Center are 
of high quality and are not subject to political influence. 

(8) The Commissioner may appoint such other ad hoc advisory 
committees as the Commissioner considers necessary. 

(dXD The Commissioner shall, not later than Jane 1 of each year, 
submit to the Congress an annual report which— 

(A) contains a description of the activities of the Center 
during the then current fiscal year and a projection of its ac- 
tivities during the succeeding fiscal year, 

(B) sets forth estimates of the cost of the projected activities 
for such succeeding fiscal year; and 

(C) includes a statistical report on the condition of education 
in the United States during the two preceding fiscal years and 
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a projection, for the three succeeding fiscal year*, of estimated 
statistics related to education in the United States. 
(2) The Secretary shall tubmit annually a report to the Congress 
giving information of the State of Education in the Nation. In such 
report the Secretary thall dearly set forth the Secretary's viewi of 
critical needs in education and the most affective manner in which 
the nation and the Federal Government may address such needs. 

(8) The Center shall develop and enforce standards designed to 
protect the confidentiality of persons in the collection, reporting, 
and publication of data under this section. This subparagraph shall 
not be construed to protect the confidentiality of information about 
institutions, organizations, and agencies receiving. ,grmt* from or 
having contracts with the Federal Government 

(4XA) 1 Except as provided in this section, no person may— 

(i) use any individually identifiable information fur* 
niahed under the provisions of this section for any purpose 
other than statistical purposes for which it is supplied; 

(ii) make any publication whereby the data furnished by 
any particular person under this section can be iden t ified; 
or 

(iii) permit anyone other than the individuals authorized 
by the Commissioner to examine the individual reports; or 

(B) No department, bureau, agency, officer, or employee of 
the Government, except the Commissioner of Education Statis- 
tics in carrying out the purposes of this section, shall require, 
for any reason, copies of reports which have been filed under 
this section with the Center for Education Statistics or re- 
tained by any individual respondent Copies of such reports 
which have been so retained or filed with the Center or any of 
its employees or contractors or agents shall be immune from 
legal process, and shall not, without the consent of the individ- 
ual concerned, be admitted as evidence or used for any purpose 
in any action, suit, or other judicial or administrative proceed- 
ing. This subsection shall only apply to individually identifia- 
ble data (as defined in subparagraph (E)\* 

(C) Whoever, being or having been an employee or staff 
member appointed under the authority of the Commissioner or 
in accordance with this section of the Act, having taken and 
subscribed the oath of office, or having sworn to observe the 
limitations imposed by subsection (a), knowingly publishes or 
communicates any individually identifiable information (as de- 
fined in subparagraph (E)), the disclosure of which is prohibit- 
ed under the provisions of subparagraph (A), and which comes 
into his or her possession by reason of employment (or other- 
wise providing services) under the provisions of this section, 
shall be fined under title 18, United States Code, or imprisoned 
not more than 5 years, or both. 

(D) The Commissioner may utilize temporary staff, including 
employees of Federal, State, or local agencies or instrumental- 
ities including local education agencies, and employees of pri- 
vate organizations to assist the Center in performing the work 
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authorized by this section, but only if such temporary staff is 
•worn to observe the limitations imposed by this section. 

(E) No collection of information or data acquisition activity 
undertaken by the Center shall be subject to any review, co- 
ordination or approval procedure except as required by the Di- 
rector of the Office of Management and Budget under the rules 
and regulations established pursuant to chapter 85 of title 44, 
United State* Code. 

(F) For the purposes of this section— 

(i) the term "individually identifiable information* 9 
means any record, response form, completed survey or ag- 
gregation thereof from which information about individual 
students* teachers, administrators or other individual per- 
sons may be revealed; 

(ii) the term "report" means a response provided by or 
about an individual to an inquiry from the Center and 
does not include a statistical aggregation from which indi- 
vidually identifiable information cannot be revealed; and 

(iii) as used in clause (i), the term "persons" does not in- 
clude States, local educational agencies, or schools. 

W3Xi) This paragraph shall not apply to— 

(I) the survey required by section 1303(c) of the Higher 
Education Amendments of 1986; or 

(II) to any longitudinal study concerning access, choice, 
persistence progress, or attainment in postsecondary edu- 
cation. 

(ii) Any person, except those sworn to observe the limitation 
of this subsection, who uses any data as described in clause (i) 
provided bv the Center, in conjunction with any other informa- 
tion or technique (including de-encryption), to identify any in- 
dividual student, teacher, administrator, or other person and 
who discloses, publishes, or uses for a purpose other than that 
for which it Nras collected, or who otherwise violates clause (i) 
or (ii) of subparagraph (A), shall be fined under title 18, United 
States Code, or imprisoned not more than 5 years, or both. 

(iii) No employee or staff member of the Center or of an in- 
stitution of higher education may be found criminally liable 
under subparagraph (C), based on a violation of subparagraph 
(A) or clause (i), if such employee or staff member has taken 
reasonable precautions, consistent with the purpose of this sec- 
tion, to ensure the confidentiality of data made available to the 
public. 

(H) Nothing in this paragraph shafl restrict the right of the 
Comptroller General of the United States to gain access to any 
reports or other records, including information identifying in- 
dividuals, in the Center's possession; except that the same re- 
strictions on disclosure that apply to the Center under sub- 
paragraphs (S) and (G) shall apply to the General Accounting 
Office 

(eXD The Center is authorized to furnish transcripts or copies of 
tables and other statistical records and make special statistical 
compilations and surveys for State and local officials, public and 

Erivate organizations, and individuals. The Center shall provide 
tate and local educatinnal agencies opportunities to suggest the 
development of particular compilations of statistics, surveys, and 
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analyses that would assist those educational agencies. The Center 
shall furnish such special statistical compilations and surveys as 
the Committees on Labor and Humim Resources and on Appropria- 
tions of the Senate and the Committees on Education and Labor 
and on Appropriations of the House of Representatives may re- 
quest. Sucn statistical compilations and survey*, other than those 
carried out pursuant to the preceding sentence, shall be made sub- 
ject to the payment of the actual or estimated cost of such wort In 
the case of nonprofit organizations or agencies, the Secretary may 
engage in joint statistical projects, the cost of which shall be shared 
equitably as determined by the Secretary: Provided^ That the pur- 
poses of such projects are otherwise authorized by law. All funds 
received in payment for work or services described in this para- 
graph shall be deposited in a separate account which may be used 
to pay directly the costs of such work or services, to repay appro- 
priations which initially bore all or part of such costs, or to refund 
excess sums when necessary. 

(2XA) The Center shall participate with other Federal agencies 
having a need for educational data in forming a consortium for the 
purpose of providing direct joint access with such agencies to all 
educational data received by the Center through automated data 
processing. The library of Congress, General Accounting Office, 
and the Committees on Labor and Human Resources and Appro- 
priations of the Senate and the Committees on Education and 
Labor and Appropriations of the Hou&e of Representatives shall, 
for the purposes of this subparagraph, be considered Federal agen- 
cies. 

(B) The Center shall, in accordance with regulations published 
for ths purpose of this paragraph, provide all interested parties, in- 
cluding public and private agencies and individuals, direct access to 
data collected by the Center for purposes of research and acquiring 
statistical information. 

(3) 1 In carrying out any authorized responsibilities under 
this section, the Commissioner may enter into contracts under 
regular competitive procedures of the Federal Government or 
otHer financial arrangements. Contracts or financial arrange- 
ments may also include sole source contracts with States, addi- 
tional institutions, organizations performing international 
studies, and associations that are nationally representative of a 
wide variety of States or nonpublic schools. The Commissioner 
shall submit annually a repor^ to the appropriate committees 
of the Congress, listing eacl source contract, its purpose, 
and the reasons why competitive bidding was not feasible in 
each such instance. 

(4) The Commissioner is authorized to prepare and publish 
such information and documents as may be of value in carry- 
ing out the purposes of this section. Periodically, the Commis- 
sioner shall issue a regular schedule of publications. 

(5} I addition to the condition of education report under 
subsection (d), the Commissioner is authorized to make special 
reports on particular subjects whenever required to do so by 
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the President or either Home of Congress or when considered 
appropriate by the Commissioner. 

(6) The Commissioner is authorized to use information col- 
lected by other offices in the Department of Education and by 
other executive agencies and to enter into interagency agree- 
ments for the collection of statistics for the purposes of this 
section. The Commissioner is authorized to arrange with any 
agency, organization, or institution for the collection of statis- 
tics for the purposes of this section and may assign employees 
of the Center to any such agency, organization, or institution 
to n&sLl in such collection. 

(7) The Commissioner is authorized to use the statistical 
method known as sampling to carry out this section. Data may 
be collected from States, local educational agencies, schools, li- 
braries, administrators, teachers, students, the general public, 
and such other individuals, persons, organizations, agencies, 
and institutions at the Commissioner may consider appropri- 
ate. 

(8) To assure the technical quality and the coordination of 
statistical activities of the Department, the Commissioner shall 
provide technical assistance to Department offices that gather 
data for statistical purposes. Sucn assistance may include a 
review of and advice on data collection plans, survey designs 
and pretests, the management of data, and the quality of re- 
porting of data. 

(9) The Commissioner is authorized to— 

(A) select, appoint, and employ such officers and em- 
ployees as may oe necessary to carry out the functions of 
the Center, subject to the provisions of title 5, United 
States Code (governing appointments in the competitive 
service), and the provisions of chapter 51 and subchapter 
III of chapter 53 of such title (relating to classification and 
General Schedule pay rates); and 

(B) notrothstanoing any other provision of this Act, to 
obtain services as authorized by section 8109 of title 5, 
United States Code, at a rate not to exceed the equivalent 
daily rate payable for grade GS-18 of the General Sched- 
ule under section 5332 of such title. 

(fXl) 1 There are authorized to be appropriated for the purposes 
of this section (including salaries ana expenses) $42,323,000 for 
fiscal year 1989, and such sums as may be necessary for each of the 
fiscal years 1990, 1991, 1992, and 1993. 

(2)* The Commissioner may contract with States to carry out 
subsection (h). Such contracts may not exceed the additional cost to 
the State— 

(A) of meeting the information and data gathering requirements 
in compliance with such subsection; or 
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(B) for compliance with related efforts of the National Center for 
Education Statistics to achieve comparable and uniform data con- 
sistent with the purpose© of this subsection. 

(gXl) In addition to its other responsibilities, the Center shall col* 
lect uniform data from the States on the financing of elementary 
and secondary education. Each State receiving funds under the 
Education Consolidation and Improvement Act of 1981 shall coop- 
erate with the Center in this effort. 

(2) 1 In addition to other duties of the Commissioner under 
this section, it ahall be the responsibility of the Commissioner 
to issue regular public reports to the President and Congress 
on dropout and retention rates, results of education, supply 
and demand of teachers and school personnel, libraries, finan- 
cial aid and on such other education indicators as the Commis- 
sioner determines to be appropriate. 

(5) The Commissioner shall establish a special study panel to 
make recommendations concerning the determination of edu- 
cation indicators for study and report under paragraph (2). Not 
more than 18 months after the date of the enactment of the 
Augustus F. Hawkin*-Robert T. Stafford Elementary and Sec* 
ondary School Improvement Amendments of 1988, the Commit 
sioner shall submit the Teport of the panel to the appropriate 
committees of the Congress. The panel shall oaase to exist 6 
months after the date o? such submission. 

(4XA) The Center shall conduct an annual national survey of 
dropout and retention rates as an education indicator. 

(B) The Commissioner shall appoint a special task force to 
develop and test an effective methodology to accurately meas- 
ure dropout and retention rates. Not later than 1 year after 
the date of enactment of the Augustus F. Hawkins-Robert T. 
Stafford Elementary and Secondary School Improvement 
Amendments of 1988, the task force shall submit a report of its 
recommendation*, including procedures for implementation of 
such recommendations, to the Commissioner and the appropri- 
ate committees of the Congress. 

(C) On the second 1 Tuesday after Labor Day of 1989 and on 
each such Tuesday thereafter, the Center shall submit a report 
to the appropriate committees of the Congress of the dropout 
and retention rate prevailing on March 80 of each such year. 

(5XA) As of March 30. 1990, and not less than every 8 years 
thereafter, the Center shall conduct a national study and 
survey of financial aid in accordance frith the provisions of sec- 
tion 1303(c) of the Higher Education Amendments of 1986. The 
Center shall submit a report to the appropriate committees of 
the Congress concerning the findings of such study* 

(6) Concurrent with each survey, the Center. 4 shall conduct 
longitudinal studies of freshman and graduating students con- 
cerning access, choice, persistence progress, curriculum and at- 
tainment. Such studies shall evaluate such students at S points 
over a 6-year interval. 
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(6) On April l t 1993, and every 10 years thereafter, the 
Center Khali submit a report to the appropriate committees of 
the Congress concerning the social ana economic status of chil- 
dren who reside in the anas served by different local educa- 
tional agencies. Such report shall be based on data collected 
during the most recent decennial census. 

(7) The Center shall conduct a study of a statistically rele- 
vant sample of students enrolled in elementary and. secondary 
school and postsecondary education training concerning educa- 
tional progress, intellectual development, and economic pros- 
perity. The study shall collect data on participation in higher 
education, including enrollment, persistence, and attainment 
Such study shall evaluate such students by such criteria at 2- 
year intervals. As of February 1, 19S9, and every 8 years there- 
after, the Commissioner shall select a sample of students en- 
rolled in school for this study. 

(8) The Center with the assistance of State library agencies, 
shall develop and support a cooperative system of annual data 
collection for jpublic libraries. Participation shall be voluntary; 
however, all States should be encouraged to join the system. 
Attention should be given to insuring 1 timely, consistent and 
accurate reporting. 

(9) The National Center for Education Statistics shall conduct a 
study on the effects of higher standards prompted by school reform 
efforts on student enrollment and persistence. The study shall ei- 
tmine academic achievement, and graduation rates of low-income, 
h&udic&pped, limited English proficient, and educationally disad- 
rrziiz^ci students. 

(hXl) There is established within the (inter a National Coopera- 
tive Education Statistics System (hereafter* referred to in this sub- 
section as the "System"). The purpose of the System is to produce 
and maintain, with the cooperation of the States, comparable and 
uniform educational information and data that are useful for pol- 
icymaking at the Federal, State, and local level. 
(2) Each State that desires to participate in the system shall— 
(A) first develop with the Center the information and data- 
gathering requirements that are needed to report on the condi- 
tion and progress of elementary and secondary education in 
the Unites States, such as information mxd data on— 

(i) schools and school districts; 

(ii) students and enrollments, inch ling special popula- 
tions; 

(iii) the availability and use of school libraries and their 
resources; 

(iv) teachers, librarians, and school administrators; 

<v) the financing of elementary and secondary education; 

(vi) student outcomes, including scores on standardized 
tests and other measures of educational achievement; and 

(vii) the progress of education reform in the States and 
the Nation; and 
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(B) then enter into en agreement with the Center for that 
fiscal year to comply with those information and data-gather* 
ing requirements. 
(3) To establish and maintain the system, 1 the Commissioner— 

(A) shall— 

(i) provide technical as, itance to the States regarding 
the collection, maintenance, and use of the System s data, 
including the timely dissemination of such data: and 

(ii) to the extent possible, implement standard defini- 
tions and data collection procedures; and 

(B) may— 

(i) directly, or through grants, cooperative apeements, 
or contracts, conduct research, development, demonstra- 
tion, and evaluation activities that are related to the pur* 
poses of the System; and 

(ii) prescribe appropriate guideline: to ensure that the 
statistical activities of the States participating in the 
System produce data that are uniform, timely, and appro- 
priately accessible. 

(iXl) With tbe advice of the National Assessment Governing 
Board established by paragraph (5XaXi), the Commissioner shall 
carry out, by grants, contracts, or cooperative agreements with 
qualified organizations, or consortia thereof, a National Assessment 
of Educational Progress. The National Assessment of Educational 
Progress shall be placed in the National Center for Education Sta- 
tistics arid shall report directly to the Commissioner for Education- 
al Statistics. The purpose of the National Assessment is the assess- 
ment of the performance of children and adults in the basic skills 
of reading, mathematics, science, writing, history/geography, and 
other areas selected by the Beard. 

(2XA) The National Assessment shall provide a fair and accurate 
presentation of educational achievement in skills, abilities, and 
knowledge in reading, writing, mathematics, science, history/geog- 
raphy, and other areas specified by the Board, and shall use sam- 
pling techniques that produce data that are representative on a na- 
tional and regional basis and on a State basis pursuant to subpara- 
graphs (CXi) and (CXii>. In addition, the National Assessment 
shall- 

(i) collect and report data on a periodic basis, at least once 
every 2 years for reading and mathematics; at least once every 
4 years for writing and science; and at least once every 6 years 
for history/geography and other subject areas selected by the 

Board; 

(ii) collect and report data every 2 years on students at ages 
9, 13, and 17 and in grades 4, 8, and 12; 

(iii) report achievement data on a basis that ensures valid re- 
liable trend reporting; 3 

(iv) include information on special groups. 

(B) In carrying out the provisions of subparagraph (A), the Secre- 
tary and the Board appointed under paragraph (5) shall assure that 
at least 1 of the subject matters in each of the 4 and 6 year cycles 
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described in subparagraph (AXi) will be included in sach 2 year 
cycle Assessment 

(CXi) The National Assessment shall develop a trial mathematics 
assessment survey instrument for the eighth grade and shall con- 
duct a demonstration of the instrument in 1990 in States which 
wish to participate, with the purpose of determining whether such 
an assessment yields valid, reliable State representative data. 

(u) The National Assessment shall conduct a trial mathematics 
assessment for the fourth and eighth grades in 1992 and, pursuant 
to subparagraph (6XD),' shall develop a trial reading assessment to 
be administered in 1992 for the fourth grade in States which wish 
to participate, with the purpose of determining whether such an as- 
sessment yields valid, reliable State representative data. 

(iii) The National Assessment shall ensure that a representative 
sample of students participate in such assessments. 

(iv) No State may apree to participate in the demonstration de- 
scribed in this subsection without full knowledge of the process for 
consensus decisionmaking on objectives to be tested, required in 
paragraph (6XE), and of assessment demonstration standards for 
sampling, test administration, test security, data collection, valida- 
tion and reporting. States wishing t participate shall sign an 
agreement developed by the Commissioner. A participating State 
shall review and give permission for release of results from any 
test of its students administered as a part of this demonstration 
prior to the release of such data. Refusal by a State to release its 
data shall not restrict the reporting of data from other States that 
have approved the release of such data, 

(v) The Commissioner shall provide for an independent evalua- 
tion conducted by a nationally recognized organization (such as the 
National Academy of Sciences or the National Academy of Educa- 
tion) of the pilot programs to aasess the feasibility and validity of 
assessments and the fairness and accuracy of the data they 
produce. The report shall also describe the technical problems en- 
countered and a description about what was learned about how to 
best report data from the National Assessment of Educational 
Progress. The results of this report will be provided to the Congress 
and to States which participated in assessments pursuant to para- 
graph (C) (i) and (ii) within 18 months of the time such assessments 
were conducted. 

(DXi)* The National Assessment shall have the authority to de- 
velop and conduct, upon the direction of the Board and subject to 
the availability of appropriations, assessments of adult literacy. 

(3XA) The National Assessment shall not collect any data that 
are not directly relsted to the appraisal of educational perform- 
ance, achievements, and traditional demographic reporting varia- 
bles, or to the fair and accurate presentation of such information. 

(B) The National Assessment shall provide technical assistance to 
States, localities, and other parties that desire to participate in the 
assessment to yield additional information described in paragraph 



(2). 
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(4XA) Except as provided in subparagraph (B), the public shall 
have access to all data, question*, and test instrument* of the Na- 
tional Assessment 

(BXi) The Commianoner shall ensure that all personally identifia- 
ble information about students, their educational performance, and 
their families and that information with respect to individual 
schools remain confidential, in accordance with section 652a of title 
5 f United States Code. 

(ii) Notwithstanding any other provision of the law, the Secretary 
may decline to make available to the public for a period not to 
exceed 10 years following their initial use cognitive questions that 
the Secretary intends to reuse in the future. 

(C) The use of National Assessment test items and test data em- 
ployed in the pilot program authorized in subsection (2XC) to rank, 
compare, or otherwise evaluate individual students, schools, or 
school districts is prohibited. 

(SXAXi) There is established the National Assessment Governing 
Board (hereafter 1 in this section referred to as the "Board"). 

(ii) The Board shall formulate the policy guidelines for the Na- 
tional Assessment 

(B) The Board shall be appointed by the Secretary in accordance 
with this subparagraph ana subparagraphs (O, (D), and (£). The 
Board shall be composed of— 

(i) two Governors! or former Governors, who shall not be 
members of the same political party; 

(ii) two State legislators, who shall not be members of the 
same political party; 

(iii) two chief State school officers; 

(iv) one superintendent of a local educational agency; 

(v) one member of a State board of education; 

(vi) one member of a local board of education; 

(vii) three classroom teachers representing the grade levels 
at which the National Assessment is conducted; 

(viii) one representative of business or industry; 

(ix) two curriculum specialists; 

(x) two testing and measurement experts; 

(xi) one nonpublic school administrator or policymaker, 

(xii) two school principals, one elementary and one second- 

**y; 

(xiii) three additional members who are representatives of 
the general public, including parents. 

The Assistant Secretary for Educational Research and Improve- 
ment shall serve as an ex officio member of the Board as a nonvot- 
ing member. 

(CXi) The Secretary and the Board shall ensure at all times that 
the membership of the Board reflects regional, racial, gender and 
cultural balance and diversity and that it exercises its independent 
judgment free from inappropriate influences and special interests. 

(ii) In the exercise of its functions, powers, and duties, the Board 
shall hire its own staff ani. shall be independent of the Secretary 
and the other offices and officers of the Department of Education. 
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(m) The Secretary may appoint, mt the direction of the Board, for 
terms not to exceed 3 yean, without regard to the provisions of 
title 5, United States Code, governing appointments in the compete 
tive service, not more than 6 technical employees to administer 
this subsection who may be paid without regard to the provisions 
of chapter 51 and subchapter m of chapter 63 of such title relating 
to classification and General Schedule pay rates. 

CDXi) The members of the Assessment Policy Committee, serving 
on the date of enactment of the National Assessment of Education* 
al Progress Improvement Act, shall become members of the Board 
for the remainder of the terms of the appointment to the Am***- 
ment Policy Committee. 

(ii) To complete the initial membership of the Board, the Secre- 
tary shall appoint members of the Board as necessary in the cate- 
gories described in subparagraph (B) for which there are no mem- 
bers continuing from the Assessment Policy Committee on the date 
of enactment of the National Assessment of Educctiorul Progress 
Improvement Act. The Secretary shall appoint such members from 
among nominees furnished by the Governors, chief State school of* 
ficers, education associations and organizations, the National Acad- 
emy of Sciences, the National Academy cf Education, parent orgs* 
nizations, and learned societies. 

(iii) As vacancies occur, new members of the Board shall be ap- 
pointed by the Secretary from among individuals who are nominat- 
ed by the Board after consultation with representatives of the 
groups listed in subparagraph (B). For each vacancy the Board 
shall nominate at least 3 individuals who, by reason of experience 
or training, are qualified in that particular Board vacancy. 

(E) Members of the Board appointed in accordance with this 
paragraph shall serve for terms not to exceed 4 years which shall 
be staggered, as determined by the Secretary, subject to the provi* 
sions of subparagraph (DXi). Any appointed member of the Board 
who changes status under subparagraph (B) during the term of the 
appointment of the member may continue to serve as a member 
until the expiration of that term. 

(6XA) In carrying out its functions under this subsection, the 
Board shall be responsible for— 

<i) selecting subject areas to be assessed (consistent with 
paragraph (2XA)); 

(ii) identifying appropriate achievement goals for each age 
and grade in each subject area to be tested under the National 
Assessment; 

(iii) developing assessment objectives; 

(iv) developing test specifications; 

(v) designing the methodology of the assessment; 

(vi) developing guidelines and standards for analysis plan* 
and for reporting and disseminating results; 

(vii) developing standards and procedures for interstate, re- 
gional and national comparisons; and 

(viii) taking appropriate actions needed to improve the form 
and use of the National Assessment 

(B) The Board may delegate any functions described in subpara- 
graph (A) to its staff. 

(C) The Board shall have final authority on the appropriateness 
of cognitive items. 
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(D) The Board thai! take steps to ensure that all items selected 
for use in the National Assessment are free from racial, cultural, 
gender, or regional bias. 

(E) Each learning area assessment shall have goal statements de- 
vised through a national consensus approach, providing for active 
participation of teachers, curriculum specialists, local school ad- 
ministrators, parents and concerned members of the general 
public. 

(F) The Secretary shall report to the Board at regular intervals 
cf the Department's action to implement the decisions of the 

Board. 

(G) Any activity of the Board or of the organization described in 
paragraph (1), shall be subject to the provisions of this subsection. 

(7XA) Not to exceed 10 percent of the funds available for this sub- 
section may be used for administrative expenses (including staff, 
consultants and contracts authorized by the Board) and to carry 
out the functions described in paragraph (6XA). 

(B) For the purposes of its administrative functions, the Board 
shall have the authorities authorized by the Federal Advisory Com- 
mittee Act and shall be subject to the open meeting provisions of 
that law. 

(8XA) Participation in the National and Regional Assessments by 
State and local educational agencies shall be voluntary. # 

(B) Participation in assessments made on a State basis shau be 
voluntary. The Secretary shall enter into an agreement with any 
State wtiich desires to carry out an assessment for the State under 
this subsection. Each such agreement shall contain provisions 
designed to assure — 

(i) that the State will participate in the assessment; 

(ii) that the State will pay from non-Federal sources the non- 
Federal share of participation; and 

(iii) that the State agrees with the terms and conditions spec- 
ified in subsection (aX2XCXiv). 

(CXi) For each fiscal year, the non-Federal share for the purpose 
of clause (ii) of subparagraph (B) shall be the cost of conducting tne 
assessment in the State including the cost of administering the as- 
sessment at the school level for all schools in the State sample and 
the cost of coordination within the State, 

(ii> The nonfederal share of payments under this paragraph 
rnav be in cash or in kind. . 

(§mA> The Commissioner shall provide for continuing reviews of 
the National Assessment, including validation studies by the Na- 
tional Center for Education Statistics and solicitation of public 
comment on the conduct and usefulness of the National Assess- 
ment. The Secretary shall report to the Congress, the President, 
and the Nation on the findings and recommendations of such re- 
views. The Commissioner shall consider the findings and recom- 
mendations in designing the competition to select the organization 
through which the Office carries out the National Assessment. 

(B) The Commissioner shall, not later than 6 months after the 
date of enactment of the National Assessment of Educational 
Progress Improvement Act, publish a report setting forth plans for 
the collection of data for the 1990 assessment and plans for includ- 
ing other subject areas in the 1992 and later assessments. The 
report shall include methods by which the results of the National 
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Assessment of Educational Progress may be reported so that the 
result* are more readily available and more easily understood by 
educators, poucymakers, and the general public, and methods by 
which items will be reviewed to identify and exclude items which 
reflect racial, cultural, gender, or regional bias. The report shall be 
developed after consultation with educators, State education offi- 
cials, members of the Board appointed under paragraph (5), and 
the general public. ' 

(C) The report required by this paragraph shall be submitted to 
the Umgress and made available to the public. The appropriate au- 
thorizing committees of the Congress may request the Secretary to 
modify the plan contained in the report. The Secretary shall take 
such actions as may be appropriate to carry out the recommenda- 
tions contained in the report. 

.J? 1 /ft PY 7 ? 08 ** °X thB •ection, the terms "United States" and 
State include the District of Columbia and Puerto Rico. 

(20 UACjSle-l) i&ugMd Aufurt 21. 1974. PX 53-380. mc. Mlu). 88 Sut. 556. 
i^'iSylt 4 WJP' 1976, PX - 9< - 273 ' 12m - 90 St *t- *W: amended October 
& ^ V* •fc?. 82, Mc - 401(c) ' 90 **»»• "c- 406. W Sut. 2231. 2232Tmc 
?2! ( . ql \ < ft§ ut ms - »=>«ded Nov. 1. 1978. PX. 95-561. Ma 1201. 121a* . (el' 

?A i rA.?^;? 8 ^ ,ec 83 Sul - CT7 : *m«nded Oct. 19. 1984. Pi. 98-511. wet 702(b) 
704/blW Sut 2406; amended Oct. 17. 1986. PX 99-496. .k. 1402. 100 Sut 1597 
WiSPiS* *• 198,1 PX 10 °- 50 ' » 24(bl - 101 s ** 1 - S 63 ; amended April 28 1988 

RESPONSIBILITY OF STATES TO FURNISH INFORMATION 

Sec. 406A. (a) The Commissioner shall require that each State 
submit to him, within ninety days after the end of any fiscal year, 
a report on the uses of Federal funds in that State under any appli- 
cable program for which the State is responsible for administra- 
tion. Such report shall — 

(1) list all grants and contracts made under such program to 
the local educational agencies and other iwblic and private 
agencies and institutions within such State during such year 

(2) include the total amount of funds available to the State 
under each such program for such fiscal year and specify from 
which appropriation Act or Acts these funds were available; 

(8) be made readily available by the State to local education- 
al agencies and other public and private agencies and institu- 
tions within the State, and to the public. 
<b) On or before March 81 of each year, the Commissioner shall 
submit to the Committee on , Labor and Human Resources of the 
benate and to the Committee on Education and Labor of the House 
of Representatives an analysis of thsce report* and a compilation of 
statistical data denved therefrom. 

(20 VS.C. 1221e-l«) Erueted Auguit 21. 1974, FX 93-880. mc. 512ui 88 Sut S71- 
amended April 21. 1976. PX. 94-273. mc. 17. 90 Sut. 379; amended 'October 12 1976 
PX. 94-482, mc. 501(fX2v<fX3). 90 Stat. 2237; nd«igns*d INov. I 1978 PX 95-561 
mc 1231UX2). 92 Sut. 2342; Me alao general ! reteen* Oct 11, 1979. PX 
801, 93 SUL 677; amended teetabir i, 1983. PX 97-21 1?mc. IB. 9 Sttt 1417 ' 
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